wiki:HowToUseBrainstormsSoftware

Version 32 (modified by sc, 12 years ago) (diff)

--

How To Use Puzzlebox Brainstorms Software

Description

Puzzlebox Brainstorms software is undergoing rapid development, including more convenient packaging and reduce hardware and software requirements. This document will be updated periodically to reflect advances in development a packaging.

The current version of Puzzlebox Brainstorms software uses an Emotiv EPOC EEG headset running software on a Windows PC to send control commands over wireless network connection (either Wifi or mobile data) to a Nokia n900 mobile handset running Linux, which in turn speaks to a LEGO Mindstorms NXT robot over Bluetooth. This configuration allows the camera on the n900 to be used to remotely view a live stream from the robot's perspective while the user tele-operates remotely.

The Nokia n900 / Maemo 5 software portion should be possible to run on any Linux PC which supports Bluetooth. With minor modification it should be possible to run entirely on the Windows PC as well, and this is a future target for development, along with removing the need for the "EmoKey" software program.

Alternatively, the "Remote Control" GUI software can be run entirely on a Nokia n900, allowing control of the robot without an EEG headset.

Equipment

Hardware

Software (Windows PC)

Software (Nokia n900 running Maemo 5 Linux)

Setup

Download

  • The latest stable and development releases of Puzzlebox Brainstorms software can be downloaded here

Configuring LEGO Mindstorms NXT

  • First select the Bluetooth Menu

No image "nxt-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next turn on to Bluetooth

No image "nxt-2-bluetooth_on_off.jpg" attached to HowToUseBrainstormsSoftware

  • Then turn on Visibility

No image "nxt-3-bluetooth_visibility.jpg" attached to HowToUseBrainstormsSoftware

Configuring Nokia n900 running Maemo 5 Linux

  • First select and turn on Bluetooth from the control panel menu

No image "n900-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next choose the LEGO Mindstorms NXT from the list of Bluetooth devices

No image "n900-2-device-list.jpg" attached to HowToUseBrainstormsSoftware

  • Then choose "Edit" and set the NXT device as "Trusted"

No image "n900-3-save_device.jpg" attached to HowToUseBrainstormsSoftware

  • Now save all Bluetooth settings
  • Next open the "X Terminal" application or SSH into the n900
  • Use the "hcitool" to scan for Bluetooth devices

No image "terminal-1-bluetooth_scan.jpg" attached to HowToUseBrainstormsSoftware

  • Note the Bluetooth MAC address of the NXT device
  • Connect the NXT device to the serial port "/dev/rfcomm0" using the Bluetooth MAC address

No image "terminal-2-bluetooth_connect.jpg" attached to HowToUseBrainstormsSoftware

  • Finally run the Puzzlebox Brainstorms Server software

No image "terminal-3-start_server.jpg" attached to HowToUseBrainstormsSoftware

Configuring Windows PC

  • Currently software installation and execution is a manual process, though an automated install package is currently in development.
  • Python 2.5.4 is the currently recommended version of Python to download and install, see link above.
  • Pygame and Python Twisted modules each need to be installed next, again all necessary download links are as above.
  • The Emotiv SDKlite software toolkit will also need to be downloaded and installed from the Emotiv website.
  • By default the Client Interface software will attempt to connect to the Server software at local IP address "192.168.1.150" on port "8194" - this setting can be changed by opening the file "puzzlebox_brainstorms_configuration.py" and modifying the fields "SERVER_HOST" and "SERVER_PORT" respectively. Be sure if changing the server port the same file is also changed on the Server side, and the Server restarted for the change to take effect.

Running Puzzlebox Brainstorms Software

  • Launch the Client Interface software file "puzzlebox_brainstorms_client_interface.py"

No image "windows-1-launch_interface.jpg" attached to HowToUseBrainstormsSoftware

  • Clicking on any of the arrows on screen will transmit driving commands to your LEGO Mindstorms NXT robot through the PC's network connection to the remote mobile computer running Linux (which in turns issues to commands via Bluetooth to the LEGO Mindstorms NXT robot itself).

No image "windows-2-drive_forward.jpg" attached to HowToUseBrainstormsSoftware

Keyboard Commands

The following keystrokes can also be used to steer the robot:

  • "[b]wb" or "[b]up arrowb" - drive forward
  • "[b]ab" or "[b]left arrowb" - drive left
  • "[b]sb" or "[b]xb" or "[b]down arrowb" - drive right
  • "[b]db" or "[b]right arrowb" - drive reverse
  • "[b]zb" - turn left in reverse
  • "[b]cb" or "[b]page downb" - turn right in reverse

Control Via Emotiv EPOC EEG Headset

EmoKey

  • The Emotiv EmoKey program is used to receive detections from the EEG Headset via the Emotiv Control Panel in realtime, or pre-scripted detections from the EmoComposer utility.
  • EmoKey translates these detections into keystrokes which are issued to the Puzzlebox Brainstorms Client Interface for steering the LEGO Mindstorms NXT robot.
  • To begin, launch the EmoKey program, and load the mapping file "puzzlebox_brainstorms.ekm" found in the "emokey" folder of the Puzzlebox Brainstorms Software.

No image "windows-3-emokey_mapping.jpg" attached to HowToUseBrainstormsSoftware

Documentation Under Construction