wiki:HowToUseBrainstormsSoftware

Version 74 (modified by sc, 12 years ago) (diff)

--

How To Use Puzzlebox Brainstorms Software

Description

Puzzlebox Brainstorms software is undergoing rapid development, including more convenient packaging and reduce hardware and software requirements. This document will be updated periodically to reflect advances in development and packaging.

The current version of Puzzlebox Brainstorms software uses an Emotiv EPOC EEG headset running software on a Windows PC to send control commands over wireless network connection (either Wifi or mobile data) to a Nokia n900 mobile handset running Maemo 5 Linux, which in turn speaks to a LEGO Mindstorms NXT robot over Bluetooth. This configuration allows the camera on the Nokia n900 to be used to remotely view a live stream from the robot's perspective while the user tele-operates remotely.

The Nokia n900 / Maemo 5 Linux software portion should be possible to run on any Linux PC which supports Bluetooth. With minor modification it should be possible to run entirely on the Windows PC as well, and this is a future target for development, along with removing the need for the EmoKey software program.

Alternatively, the "Remote Control Client Interface" GUI software can be run entirely on a Nokia n900, allowing control of the robot without an EEG headset.

Equipment

Hardware

Software (Windows PC)

Software (Nokia n900 running Maemo 5 Linux)

Setup

Download

  • The latest stable and development releases of Puzzlebox Brainstorms software can be downloaded here

Configuring LEGO Mindstorms NXT

  • First select the Bluetooth Menu

No image "nxt-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next turn on Bluetooth

No image "nxt-2-bluetooth_on_off.jpg" attached to HowToUseBrainstormsSoftware

  • Then turn on "Visibility"

No image "nxt-3-bluetooth_visibility.jpg" attached to HowToUseBrainstormsSoftware

Configuring Nokia n900 running Maemo 5 Linux

  • First select and turn on Bluetooth from the control panel menu

No image "n900-1-bluetooth_menu.jpg" attached to HowToUseBrainstormsSoftware

  • Next choose the LEGO Mindstorms NXT from the list of Bluetooth devices

No image "n900-2-device-list.jpg" attached to HowToUseBrainstormsSoftware

  • Then choose "Edit" and set the Lego Mindstorms NXT device as "Trusted"

No image "n900-3-save_device.jpg" attached to HowToUseBrainstormsSoftware

  • Now save all Bluetooth settings

Configuring Windows PC

  • Currently software installation and execution is a manual process, though an automated install package is currently in development.
  • Python 2.5.4 is the currently recommended version of Python to download and install, see link above.
  • Pygame and Python Twisted modules each need to be installed next, again all necessary download links are as above.
  • The Emotiv SDKlite software toolkit will also need to be downloaded and installed from the Emotiv website.
  • By default the Client Interface software will attempt to connect to the Puzzlebox Brainstorms Server software running on the Nokia n900 at local IP address "192.168.1.150" on port "8194". These setting can be changed by opening the file "puzzlebox_brainstorms_configuration.py" and modifying the fields "SERVER_HOST" and "SERVER_PORT" respectively. Be sure if changing the server port on the Puzzlebox Brainstorms Client the "puzzlebox_brainstorms_configuration.py" file is also changed on the Nokia n900. If running, the Puzzlebox Brainstorms Server must be restarted for the change to take effect.

Running Puzzlebox Brainstorms Software

Nokia n900 running Maemo 5 Linux

Server

  • Open the "X Terminal" application or SSH into the Nokia n900
  • Use the "hcitool" to scan for Bluetooth devices

No image "terminal-1-bluetooth_scan.jpg" attached to HowToUseBrainstormsSoftware

  • Note the Bluetooth MAC address of the Lego Mindstorms NXT device
  • Use the "hcitool" to connect the NXT device to the new serial port "/dev/rfcomm0" using the Bluetooth MAC address of the Nokia n900.

No image "terminal-2-bluetooth_connect.jpg" attached to HowToUseBrainstormsSoftware

  • Finally run the Puzzlebox Brainstorms Server software, "puzzlebox_brainstorms_server.py"

No image "terminal-3-start_server.jpg" attached to HowToUseBrainstormsSoftware

Client Interface

Control Via Emotiv EPOC EEG Headset

Remote Live Video Streaming From LEGO Mindstorms NXT Robot's Perspective

Currently a combination of mplayer and VNC is used to remotely view a live video stream from the LEGO Mindstorms NXT robot's perspective. mplayer is used to locally display a live video feed on the Nokia n900's screen, which is then viewed remotely over the local Wifi network via VNC. This is neither an ideal nor efficient method, but has been demonstrated to work better over a local Wifi connection than a purely Internet-based solution such as Qik.

  • First launch the "x11vnc" application on the Nokia n900

No image "n900-6-x11vnc.jpg" attached to HowToUseBrainstormsSoftware]

  • Next connect to Nokia n900 using the VNC Viewer application from a computer such as a Windows PC.
  • Slide the camera lens cover open on the rear of the Nokia n900, then exit the "Camera" application which launches automatically, making sure to leave the lens cover open.
  • Launch a new "X Terminal" application or SSH into the Nokia n900
  • Run the following mplayer command: "mplayer -vo x11 -fs tv:// -tv driver=v4l2:device=/dev/video0" (note that changing "/dev/video0" to "/dev/video1" will switch to displaying the forward-facing camera, though its resolution is considerably lower and the image quality is comparably quite poor)

No image "n900-7-mplayer_camera.jpg" attached to HowToUseBrainstormsSoftware

  • A live view from the LEGO Mindstorms NXT robot's perspective should now be visible in the VNC Viewer application.

No image "n900-8-remote_live_streaming.jpg" attached to HowToUseBrainstormsSoftware