Table of Contents

Controlling a PC from Blocks

This application note describes how you can control a target PC computer from Blocks. While this method can be used to control most aspects of the target PC (or multiple target computers for that matter), this application note discusses the following functions:

:!: While this application note is written to work with a Windows PC as the target computer, the mechanism and software used is cross platform, and will run on Windows, Mac and Linux computers.

Installation

You need two computers and a network to use this application note.

Prepare the target PC:

  1. Download and install Java (version 1.8 or later required).
  2. We have prepared a PIXILAB-blocks-root folder containing everything you need to run the examples. Download this ZIP file and unpack it. If you don't know how to do this, read the instructions in the general setup section.
  3. Inside the UIROBOT folder, launch the file named "uirobot.bat" by double-clicking it.

A command window will appear, with some text indicating that the UIROBOT program has started. You may also get a firewall alert, prompting you to allow this Java program to access the network.

If you have done the general setup, copied the files and pointed to this application notes root block, follow the next steps to run this application on your computer.

  1. Start Blocks.
  2. Open the editor using the Admin button.
  3. Log in using the appropriate credentials (default is name admin and the password pixi).
  4. On the Manage page in Blocks, select "TCP/UDP Devices".
  5. Select the "ControlledPC" device in the list.
  6. Enter the IP address of the target PC (use the ipconfig command in a new terminal window on the taget PC to find its IP address if you don't know – use a wired network).
  7. The indicator next to "ControlledPC" should turn green shortly, indicating that Blocks has connected to the UIRobot program running on the target PC.

Running the Example

Now, you're ready to start controlling the target PC. The enclosed example shows how to switch the target computer between acting as a Display Spot for Blocks or running Powerpoint in full screen mode.

Launching Chrome as a Display Spot

  1. Make sure the target PC has Google Chrome installed (if not, download and install it).
  2. Select the Task page in Blocks.
  3. Select the Main realm from the menu at the top of the window.
  4. Select the Blocks group in the list top left.
  5. Select the Start task.

If your target PC runs the 32 bit version of Windows, you may need to adjust the first two statements, setting the program and workingDir variables, to read as shown below.

The top level program folder name varies with the version of operating system and program installed. If you have a shortcut to Chrome on your desktop, you can right-click that and choose Properties to see the correct path to the Chrome application.

Finally, adjust the IP address shown at the end o the programArgs variable to match the IP address of your Blocks server, as well as the port number (using the port number shown in the Blocks editor URL bar).

The programArgs variable specifies the arguments passed to Chrome, which in this case will launch it in full screen mode, acting as a Display Spot for Blocks. Run the Start task now to launch Chrome in this way. The first time you launch Chrome you likely need to add or assign it to a display spot:

  1. Switch to the Display page of Blocks.
  2. Double-click the Display Spot named "Spot".
  3. On the Reassign dropdown list, select the ID shown in the Chrome window.
  4. Click OK.

This will show the example block assigned to the spot.

Launching and Controlling Powerpoint

Assuming you have Powerpoint installed on the target PC, do as follows to launch the powerpoint presentation enclosed with this example.

  1. Select the Task page in Blocks.
  2. Select the Main realm from the menu at the top of the window.
  3. Select the Powerpoint group in the list top left.
  4. Select the Start task.
  5. Make sure the path specified for the program variable leads to the Powerpoint application, as installed on your PC.
  6. Adjust the programArgs variable to point to the coolpresentation.pptx example powerpoint presentation included with this example.
  7. Run the Start task to launch Powerpoint running the enclosed presentation in full screen mode.

Launching another program in this way automatically terminates the previous program, in this case Chrome. Thus, this method can be used to switch between programs and to control the currently running program.

Sending keystrokes

Run the "StepNext" and "StepPrevious" task to move to the next/previous powerpoint slide. This uses the keyDown function of the UIRobot driver, which allows you to send any keystroke to the foreground program. For regular characters and numbers, you can specify the character (in upper case). For other keys, use the corresponding "virtual key" name, beginning with VK_, as listed here.

You can specify one or more modifier keys using the modifier key name followed by a plus sign:

Here are some examples:

Controlling the Mouse

While not shown in the enclosed examples, the UIRobot driver and control program also has functions and properties for moving the mouse and pressing mouse buttons, allowing you to simulate such actions

Turning Power On/Off

:!: Please note that later versions of UIrobot driver has a built in power property that handles power on/off. so the method used in this application note is somewhat outdated.

To power down the target computer, select the System task group then run the ShutDown task. This launches the shutdown.exe Windows program, which shuts down the computer.

Assuming your computer has wake-on-LAN enabled, you can turn it back on by running the StartUp task. If nothing happens when you run this command, you may need to enable wake-on-LAN in your computer's BIOS and/or in Windows.

Auto-starting the Control Program

If you're using this solution in an unattended application you probably want the UIRobot control program to start automatically whenever the target computer is powered up. Do as follows:

  1. Add a shortcut to the file named "uirobot.bat" to startup folder.
  2. Right-click the shortcut and select Properties.
  3. Set the shortcut's Run mode to "Minimized".

The next time you power up the target computer, the UIRobot control program will start automatically, ready to receive command from Blocks.

:!: IMPORTANT: The UIRobot control program can launch any program or command that a local user can run, and can simulate keystrokes and mouse clicks. Anyone that can reach the target computer over the network can perform those actions. The program has no password or other security. Only use it on a private network that you trust.

Controlling Mac/Linux Computers

The UIRobot control program also works on Mac and Linux computers if Java version 8 or later is available. Note, however, that the commands used in the enclosed example are Windows specific, and other commands and paths to programs need to be used for Mac/Linux.

Behind the Scenes

The functionality described in this application note is based on two key components:

  1. A network device driver for Blocks, named UIRobot, accepting commands from user scripts, tasks and buttons, forwarding those to the target computer.
  2. A program, also named UIRobot, running on the target computer. This program receives commands over the network.

The UIRobot driver lives in the script/driver directory inside the Blocks root. The driver has plenty of comments discussing its various functions, that you may find useful if you want to build other solutions based on this example.