wiki:HowToUseBrainstormsSoftware

Version 46 (modified by sc, 12 years ago) (diff)

--

How To Use Puzzlebox Brainstorms Software

Description

Puzzlebox Brainstorms software is undergoing rapid development, including more convenient packaging and reduce hardware and software requirements. This document will be updated periodically to reflect advances in development a packaging.

The current version of Puzzlebox Brainstorms software uses an Emotiv EPOC EEG headset running software on a Windows PC to send control commands over wireless network connection (either Wifi or mobile data) to a Nokia n900 mobile handset running Maemo 5 Linux, which in turn speaks to a LEGO Mindstorms NXT robot over Bluetooth. This configuration allows the camera on the Nokia n900 to be used to remotely view a live stream from the robot's perspective while the user tele-operates remotely.

The Nokia n900 / Maemo 5 Linux software portion should be possible to run on any Linux PC which supports Bluetooth. With minor modification it should be possible to run entirely on the Windows PC as well, and this is a future target for development, along with removing the need for the "EmoKey" software program.

Alternatively, the "Remote Control Client Interface" GUI software can be run entirely on a Nokia n900, allowing control of the robot without an EEG headset.

Equipment

Hardware

Software (Windows PC)

Software (Nokia n900 running Maemo 5 Linux)

Setup

Download

  • The latest stable and development releases of Puzzlebox Brainstorms software can be downloaded here

Configuring LEGO Mindstorms NXT

  • First select the Bluetooth Menu

No image "nxt-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next turn on Bluetooth

No image "nxt-2-bluetooth_on_off.jpg" attached to HowToUseBrainstormsSoftware

  • Then turn on "Visibility"

No image "nxt-3-bluetooth_visibility.jpg" attached to HowToUseBrainstormsSoftware

Configuring Nokia n900 running Maemo 5 Linux

  • First select and turn on Bluetooth from the control panel menu

No image "n900-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next choose the LEGO Mindstorms NXT from the list of Bluetooth devices

No image "n900-2-device-list.jpg" attached to HowToUseBrainstormsSoftware

  • Then choose "Edit" and set the Lego Mindstorms NXT device as "Trusted"

No image "n900-3-save_device.jpg" attached to HowToUseBrainstormsSoftware

  • Now save all Bluetooth settings

Configuring Windows PC

  • Currently software installation and execution is a manual process, though an automated install package is currently in development.
  • Python 2.5.4 is the currently recommended version of Python to download and install, see link above.
  • Pygame and Python Twisted modules each need to be installed next, again all necessary download links are as above.
  • The Emotiv SDKlite software toolkit will also need to be downloaded and installed from the Emotiv website.
  • By default the Client Interface software will attempt to connect to the Puzzlebox Brainstorms Server software running on the Nokia n900 at local IP address "192.168.1.150" on port "8194". These setting can be changed by opening the file "puzzlebox_brainstorms_configuration.py" and modifying the fields "SERVER_HOST" and "SERVER_PORT" respectively. Be sure if changing the server port on the Puzzlebox Brainstorms Client the "puzzlebox_brainstorms_configuration.py" file is also changed on the Nokia n900. If running, the Puzzlebox Brainstorms Server must be restarted for the change to take effect.

Running Puzzlebox Brainstorms Software

Nokia n900 running Maemo 5 Linux

Server

  • Open the "X Terminal" application or SSH into the Nokia n900
  • Use the "hcitool" to scan for Bluetooth devices

No image "terminal-1-bluetooth_scan.jpg" attached to HowToUseBrainstormsSoftware

  • Note the Bluetooth MAC address of the Lego Mindstorms NXT device
  • Use the "hcitool" to connect the NXT device to the new serial port "/dev/rfcomm0" using the Bluetooth MAC address of the Nokia n900.

No image "terminal-2-bluetooth_connect.jpg" attached to HowToUseBrainstormsSoftware

  • Finally run the Puzzlebox Brainstorms Server software, "puzzlebox_brainstorms_server.py"

No image "terminal-3-start_server.jpg" attached to HowToUseBrainstormsSoftware

Client Interface

  • Open a new "X Terminal" application or SSH into the Nokia n900
  • Run the Puzzlebox Brainstorms Client Interface softare, "puzzlebox_brainstorms_client_interface.py"

No image "n900-4-start_client_interface.jpg" attached to HowToUseBrainstormsSoftware

  • The Client Interface will appear on the Nokia n900

No image "n900-5-client_interface.jpg" attached to HowToUseBrainstormsSoftware

Windows PC

  • Launch the Client Interface software file "puzzlebox_brainstorms_client_interface.py"

No image "windows-1-launch_interface.jpg" attached to HowToUseBrainstormsSoftware

Touchscreen / Mouse Control

  • Clicking on any of the arrows on screen will transmit driving commands to your LEGO Mindstorms NXT robot through the PC's network connection to the remote mobile computer running Maemo 5 Linux (which in turns issues to commands via Bluetooth to the LEGO Mindstorms NXT robot itself).

No image "windows-2-drive_forward.jpg" attached to HowToUseBrainstormsSoftware

Keyboard Commands

The following keystrokes can also be used to steer the robot:

  • "w" or "up arrow" - "drive forward"
  • "a" or "left arrow" - "drive left"
  • "s" or "x" or "down arrow" - "drive right"
  • "d" or "right arrow" - "drive reverse"
  • "z" - "turn left in reverse"
  • "c" or "page down" - "turn right in reverse"

Control Via Emotiv EPOC EEG Headset

EmoKey

  • The Emotiv EmoKey program is used to receive detections from the EEG Headset via the Emotiv Control Panel in realtime, or pre-scripted detections from the EmoComposer utility.
  • EmoKey translates these detections into keystrokes which are issued to the Puzzlebox Brainstorms Client Interface for steering the LEGO Mindstorms NXT robot.
  • To begin, launch the EmoKey program, and load the mapping file "puzzlebox_brainstorms.ekm" found in the "emokey" folder of the Puzzlebox Brainstorms Software.

No image "windows-3-emokey_mapping.jpg" attached to HowToUseBrainstormsSoftware

Cognitiv detections are mapped to the following keystrokes and driving commands:

  • "Push" - mapped to "w" key - "drive forward"
  • "Pull" - mapped to "c" key - "turn right in reverse"
  • "Lift" - mapped to "w" key - "drive forward"
  • "Left" - mapped to "a" key - "drive left"
  • "Drop" - mapped to "s" key - "drive reverse"
  • "Right" - mapped to "d" key - "drive right"

Documentation Under Construction