The progress is constant, and the developments in the automated learning and artificial intelligence field have advanced. Though, no matter how hard we try to create artificial intelligence, we are always restricted by our knowledge of the human brain and intelligence.
To date, our brain is still a mystery, despite the fact that we have learned to measure and “fix” it in many aspects; it is still a “Pandora’s box” which we would like to open, but the consequences are frightening. Our brain is the most powerful CPU in the world, and its potential is without limits.
VARTEQ team, as all the engineers, love puzzles, and when we have an opportunity to work on something interesting, we immediately start acting. In this article, we shall talk about managing devices with our brain waves, we shall analyze NeuroSky MindWave Mobile device, and write a couple of applications for this device.
Interesting facts about the brain
- • The speed of signals in the human nervous system reaches 288 km/h.
- • The brain is able to keep from 3 to 1,000 terabytes of information. For example, the British National Archives describing 900 years of history has just 70 terabytes.
- • The brain consists of water by 80%.
- • The intellectual work does not tire the brain.
- • To make the brain work properly, it is necessary to drink sufficient amount of fluids
- • It is easier for brain to understand speech of men than the one of women
- • Regular prayer reduces the respiratory frequency and normalizes the brain wave oscillations, contributing to the process of the organism self-healing
- • The more educated a person is, the less brains disorders are likely to take place. The intellectual activity causes generation of additional tissue, compensating the damaged one
- • Taking on an unfamiliar activity is the best way to develop the brain. Communication with those who excel you in terms of intellectual capacities is also an efficient way to boost your brain.
Brain as a user-friendly interface for interacting with the devices
We have been accustomed to the keyboard and mouse, considering them as the most effective device to interact with a computer, as well as the touch screens in mobile devices. Now, there are many devices designed for work, like Tony Stark from “Iron Man” had, for example, set-top boxes games began the transition to contactless interfaces long time ago. It is a bit more complicated to accomplish with the computers since there are a lot of conservatives.
And what if an interface appears designed to work with a PC or smart houses using our brain waves? It would be cool and easy! But we have just started to move in this direction, and currently the device reminds rather a toy than a full-fledged way of interaction
NeuroSky Mind Wave
There are a lot of reviews on this device on the Internet, thus, we shall not go into too much details. In general, it is a mini-EEG that is adapted to read the commands from our brain.
The encephalogram is a method of measuring the brain’s activity. MindWave enables not only EEG (electric encephalogram). Let’s look at what this device gives us.
- • The eSence metrics are proprietary metrics of the NeuroSky company that measures two parameters: Attention and Meditation. These two parameters are two integers that range from 1 to 100.
- • Attention indicates the focus and the increased concentration of the user’s attention. Distractions, wandering thoughts, lack of attention, or anxiety may reduce this indicator
- • Meditation indicates the level of calmness and relaxation of the user, but it is not about physical relaxation, but the brain’s one. Distractions, wandering thoughts, anxiety, agitation, and external factors may reduce this indicator
- • Raw Data represent the raw data
- • Brainwave Bands are the indicators of Alpha, Beta, Delta, Theta brainwaves
- • Signal Quality
A number of applications demonstrating the capabilities of the device are offered along with the device. We shall not describe them in details because there are a lot of descriptions hereof on the Internet.
Application development for MindWave
Let us move to the most interesting, i.e. to the applications development for NeuroSky MindWave. On the official website, there is access to free SDK for PC (.NET), Mac, iOS, Android: http://store.neurosky.com/collections/developer, and there is a port at Unity3D 5: https://github.com/selectgithub/NeuroSkyUnityThinkGearPlugins
Application development for iOS
- First of all, we download the SDK for iOS: https://store.neurosky.com/products/ios-developer-tools-4
- We find the test project inside: ThinkGear-SDK-for-iOS-2015-11-27/Sample Project/ThinkGearTouch
- We run ThinkGearTouch.xcodeproj
- We compile the project
- Contrary to the Mac version, there are no problems with iOS, and the project is finished up easily.
Inside the SDK, there is a thinkgear_ios_api_reference.pdf file that contains quite detailed description of the API for interaction with the device, which is a big advantage.
In general, the interaction is simple and usual: you must first sign up for a class of events (delegate):
- (void)applicationDidFinishLaunching:(UIApplication *)application {
NSUserDefaults * defaults = [NSUserDefaults standardUserDefaults];
TGAccessoryType accessoryType = (TGAccessoryType)[defaults integerForKey:@"accessory_type_preference"];
BOOL rawEnabled = [defaults boolForKey:@"raw_enabled"];
if(rawEnabled) {
// setup the TGAccessoryManager to dispatch dataReceived notifications every 0.05s (20 times per second)
[[TGAccessoryManager sharedTGAccessoryManager] setupManagerWithInterval:0.05 forAccessoryType:accessoryType];
} else {
[[TGAccessoryManager sharedTGAccessoryManager] setupManagerWithInterval:0.2 forAccessoryType:accessoryType];
}
// set the root UIViewController as the delegate object.
[[TGAccessoryManager sharedTGAccessoryManager] setDelegate:(RootViewController *)[[navigationController viewControllers] objectAtIndex:0]];
[[TGAccessoryManager sharedTGAccessoryManager] setRawEnabled:rawEnabled];
[window addSubview:[navigationController view]];
// 这是我写的
RootViewController * rootVC = [[RootViewController alloc]initWithStyle:UITableViewStylePlain];
self.window.rootViewController = rootVC; // 这是我写的
[window makeKeyAndVisible];
}
In our case, this is RootViewController class; it should contain three methods:
// TGAccessoryDelegate protocol methods
- (void)accessoryDidConnect:(EAAccessory *)accessory;
- (void)accessoryDidDisconnect;
(void)dataReceived:(NSDictionary *)data;
And with the interval specified when signing up for an event – in this case 0.05 or 0.2, depending on the raw_enabled settings – dataReceived method will be used
- (void)dataReceived:(NSDictionary *)data {
//[data retain];
NSLog(@"%@",data);
NSString * temp = [[NSString alloc] init];
NSDate * date = [NSDate date];
rawValue = [[data valueForKey:@"raw"] shortValue];
temp = [temp stringByAppendingFormat:@"%f: Raw: %dn", [date timeIntervalSince1970], rawValue];
if([data valueForKey:@"blinkStrength"])
blinkStrength = [[data valueForKey:@"blinkStrength"] intValue];
// check to see whether the eSense values are there. if so, we assume that
// all of the other data (aside from raw) is there. this is not necessarily
// a safe assumption.
if([data valueForKey:@"eSenseAttention"]){
poorSignalValue = [[data valueForKey:@"poorSignal"] intValue];
//temp = [NSString stringWithFormat:@"poorsignal: %02xn", poorSignalValue];
temp = [temp stringByAppendingFormat:@"%f: Poor Signal: %dn", [date timeIntervalSince1970], poorSignalValue];
eSenseValues.attention = [[data valueForKey:@"eSenseAttention"] intValue];
eSenseValues.meditation = [[data valueForKey:@"eSenseMeditation"] intValue];
temp = [temp stringByAppendingFormat:@"%f: Attention: %dn", [date timeIntervalSince1970], eSenseValues.attention];
temp = [temp stringByAppendingFormat:@"%f: Meditation: %dn", [date timeIntervalSince1970], eSenseValues.meditation];
eegValues.delta = [[data valueForKey:@"eegDelta"] intValue];
eegValues.theta = [[data valueForKey:@"eegTheta"] intValue];
eegValues.lowAlpha = [[data valueForKey:@"eegLowAlpha"] intValue];
eegValues.highAlpha = [[data valueForKey:@"eegHighAlpha"] intValue];
eegValues.lowBeta = [[data valueForKey:@"eegLowBeta"] intValue];
eegValues.highBeta = [[data valueForKey:@"eegHighBeta"] intValue];
eegValues.lowGamma = [[data valueForKey:@"eegLowGamma"] intValue];
eegValues.highGamma = [[data valueForKey:@"eegHighGamma"] intValue];
rawCount = [[data valueForKey:@"rawCount"] intValue];
copyPoorSignalValue = poorSignalValue;
copyEEGValues = eegValues;
copySenseValues= eSenseValues;
copyRawCount = rawCount;
}
copyRawValue = rawValue;
copyBlinkStrength = blinkStrength;
if(logEnabled) {
[output release];
output = [NSString stringWithString:temp];
[self performSelectorOnMainThread:@selector(writeLog) withObject:nil waitUntilDone:NO];
}
[temp release];
// release the parameter
//[data release];
}
Application development for Unity3D 5
It is convenient to use Unity3D to write games, but for MindWave, there are plugins with ports https://github.com/selectgithub/NeuroSkyUnityThinkGearPlugins
- Download the repository https://github.com/selectgithub/NeuroSkyUnityThinkGearPlugins/archive/master.zip
- Create a new project in Unity3D
- Copy the archive content in the new project
- Set the platform to Android or iOS (currently only these options are supported)
- In this example, we create a project based on iOS: press Build and choose a place where Xcode project will be created, then save it
- Add two libraries into the project: ExternalAccessory.framework and Accelerate.framework
- Add a line in Info.plist, and then select “Support external accessory protocols” in the key, herewith, in this case you must choose from the list rather than insert a text; then add “com.neurosky.thinkgear”
- In the Build Options, disconnect BitCode: Enable Bitcode = NO
- Now, you can finish up the project, and everything will work; just for your information, we provide video of the project assembly.
To get the data, you need to add a component with the script ThinkGearController and subscribe to the events:
void Start () {
controller = GameObject.Find("ThinkGear").GetComponent();
controller.UpdateRawdataEvent += OnUpdateRaw;
controller.UpdatePoorSignalEvent += OnUpdatePoorSignal;
controller.UpdateAttentionEvent += OnUpdateAttention;
controller.UpdateMeditationEvent += OnUpdateMeditation;
controller.UpdateDeltaEvent += OnUpdateDelta;
controller.UpdateThetaEvent += OnUpdateTheta;
controller.UpdateHighAlphaEvent += OnUpdateHighAlpha;
controller.UpdateHighBetaEvent += OnUpdateHighBeta;
controller.UpdateHighGammaEvent += OnUpdateHighGamma;
controller.UpdateLowAlphaEvent += OnUpdateLowAlpha;
controller.UpdateLowBetaEvent += OnUpdateLowBeta;
controller.UpdateLowGammaEvent += OnUpdateLowGamma;
controller.UpdateBlinkEvent += OnUpdateBlink;
}
In this article we analyzed how to read metrics from the device. In the next part, we shall do more interesting things.
To be continued…
Author: Denis Panaskin