Logging Data to SD Cards

Logging telemetry data from robots is invaluable for troubleshooting and prototyping.

The microSD card provides an easy and cheap basis for microcontroller data log storage. Here's how to interface it to your microcontroller.

Serial Peripheral Interface (SPI)

SD/MMC cards (including microSD) use a fast, 4-wire, Serial Peripheral Interface or SPI bus supported by many microcontrollers.

The Arduino's AVR chips all have SPI capability in hardware and so does the mbed (ARM Cortex M3) I've been working with. PIC microcontrollers (I'm no expert) apparently have a Synchronous Serial Port (SSP) that supports SPI.

Four wires are used. !CS is an active low chip select so that you can select specific SPI devices that share the same bus. I didn't need to do that with the SD card this time around. MOSI is Master Out, Slave In. Master is the MCU, slave is the SD card. MISO is Master In, Slave Out.

I find the SPI nomenclature far more intuitive than the mind-bending RX/TX and DTE/DCE craziness of RS-232 (etc). It always takes a second to remember: do I attach RX to RX or RX to TX? And is it RX from the perspective of the pc or the micro? Gah!

Not so with SPI. Just connect MOSI to MOSI, MISO to MISO and SCK to SCK. Simple.

SD/MMC Interface

SD/MMC cards safely operate at 3.3V so interfacing to a 5V MCU is a little more complex than just wiring. More on that in a moment.

Wiring
One can buy a breakout board such as the one from Sparkfun to use microSD cards or buy a connector from DigiKey, Mouser, etc., and wire it up to a custom PCB.

The ultimate in jumper-free convenience

I designed a board that simply plugs in next to your breadboard Arduino clone, ready to go. No jumpers required. All pins are labeled both SPI function and Arduino pin. It's 5V-safe with a level shifter, and has an onboard regulator. For sale on Tindie.



Another cheap, simple approach that I discovered online is to solder 0.100" pin headers onto an SD-to-MicroSD adapter (like this one), wire it up on a breadboard or whatever, and then plug microSD cards into the adapter. See the pic below for the SD card with the pin headers soldered on.

The full size SD/MMC card or adapter has several contacts on the back. Here is the pinout:

3.3V MCU Devices

If you have a 3.3V MCU, then connect MOSI to your MCU's MOSI pin, MISO to MISO, SCK to SCK. You can either attach !CS to a pin on the MCU, to a mux (like a 74*151), or hardwire it to ground if it's the only thing on the SPI bus.

5V MCU Devices

Voltage level shifting from 5V down to the SD card's 3.3V can be done in several ways. A simple resistor voltage divider seems to be adequate but could limit data rate.  You really only need to level shift the lines going from the MCU to the SD card: MOSI and SCK. Most MCUs should read the correct levels on the 3.3V MISO line.

Here's a good tutorial with some other interfacing options for level shifting. My eeZee MicroSD above has a 3.3V regulator and level shifter IC built-in so it's 5V safe.

A couple of other possibilities are the use of level shifting chips. The Propeller Robot Control Board uses bi-directional Texas Instruments TXB0108 chips, for example. You can also use a 74*244 or 74*245.

Software Interface

Once the card is wired up, you can't just throw data at it. The card has to be formatted with a FAT16 or FAT32 filesystem and the Arduino has to run FAT filesystem code to create/append files. That is, if you want to pull data off the card using your PC or Mac.

FAT16 can only handle volumes up to 2G in size. Arduino users can incorporate the small-footprint fat16lib or SDuFAT library for use with these smaller cards. When formatting the card on XP, select "FAT" as the filesystem type.

I discovered after some testing that I had to use FAT32 software, and a larger Arduino FAT library, sdfatlib, with my 4G card.

Case Study

So why am I so interested in data logging all of a sudden?  I've entered the Sparkfun Autonomous Vehicle Competition (AVC) for 2011. The design for my robot's navigation system combines a rate gyro sensor and GPS, and likely other sensors, to be fused algorithmically.

I want to collect gyro and GPS data from real test runs and then in the comfort of my home, write prototype code to process the data to estimate heading and position. I'd previously interfaced the rate gyro and the GPS to the Arduino but I needed a cheap and easy way to store the data for later download and analysis. Using an SD card was ideal.

For my project, the Sparkfun AVC robot, I found an SD card and adapter on sale at Radio Shack and I breadboarded everything, drawing 3.3V from the SMT regulator I was already using to power the LISY300AL gyro.

I used a simple voltage divider on the !CS, MOSI, and SCK lines, consisting of 330 ohm and 180 ohm resistors (sacrificing power consumption for speed, I suppose; 5V/500 ohms = ~10mA)

I added a couple of indicator LEDs for power and data transmission. The data LED indicators are powered with small signal BJTs, the bases of which are connected to MISO and MOSI, respectively, through 1K current limiting resistors.

After some testing on the bench, it was time for the maiden voyage.  I had to mount all the electronics onto the RC truck chassis: SD card breadboard, Arduino, and GPS. Then I had to merge prototype code for the gyro, gps, and FAT32 filesystem. Then, finally, drive the RC Truck around and gather some data to analyze.

More on the data capture and analysis next time.

Interfacing RS-232 GPS to Arduino

If you missed prior posts, I have entered the 2011 Sparkfun Autonomous Vehicle Competition (AVC).

For reliable navigation around the Sparkfun building in Boulder, CO, I plan to equip the RC truck I purchased with a GPS, interfacing it to a microcontroller.

Garmin eTrex Legend

It so happens I have a Garmin eTrex Legend that I can use for prototyping until I buy a much better GPS module  for the robot. Like most GPS units, the Legend outputs NMEA 0183 serial data using RS-232 protocol. RS-232 uses +/-12V signals that will burn your average microcontroller's input pins to a crisp.

One can use a MAX232 to shift levels to MCU-friendly 0-5V or 0-3.3V but why dink around with that when a transistor, a few resistors, a prototype perfboard, and a few minutes with a soldering iron will do the job.

Some time ago I made a level shifter device to interface various microcontrollers with my PC. Since the GPS only spits out data and doesn't take commands from the MCU, the receive (RX) side of the level shifter circuit is all that's required to interface the Legend GPS.  The RX half of the circuit eliminates the D1 diode and C1 capacitor, and the TX circuit: Q1 transistor and R1, R2 resistors.  Leaving Q2, R3, R4, and R5.

Ditch the diode (D1) and capacitor (C1) and TX circuits

I soldered everything together on a Radio Shack copper-clad perfboard and added LEDs for power (green) and RX (red), then soldered the wires to a DB9 female connector so I could use the original Garmin cable.

Simple transistor level shifter

Incidentally, I discovered that the odd-ball Garmin GPS connector is available as a standalone part from this site. I ordered a pair and got them within a week or two. They're provided sort of like shareware. Get product first, try it, then pay. The product is really quite good, so I'll be sending a check soon. I swapped this custom connector in place of the DB9.

To read GPS data bytes, I used the NewSoftSerial library. I found that NSS is incompatible with newer Arduino Servo library routines so I had to disable those for now.  I plugged in the GPS and used a simple demo program to read in the NMEA sentences from the GPS and spit them out to the PC.

To parse the NMEA 0183 protocol, I used the TinyGPS library and modified the code to grab some additional information about the quality of the position fix. Namely, the number of satellites being tracked, and the horizontal dilution of position information.

The fix information will be helpful in prototyping a Kalman Filter to obtain the best heading and position estimates from the robot's GPS and rate gyro. In short, the algorithm will be able to establish an appropriate level of confidence in the GPS data. But more on that another time.

Now, if only there were some way to store the captured data for later analysis...

(Well, there is, but you have to wait for the upcoming article...)

Sparkfun AVC 2011: It's Official!

This morning, Sparkfun announced the 2011 Autonomous Vehicle Competition!

April 23, 2011 from 8:30am - 5:00pm. Races start at 9:30am sharp.

With shaking fingers, I registered on my smartphone in the middle of the airport, minutes after the announcement was posted... making me almost certainly the first person to sign up :)

The clock is ticking. Just 3 months to finish a robot car that can navigate around a building fraught with GPS-obscuring urban canyons and car-destroying stealth curbs... while also dodging obstacles along the way.

That's right. the on-land robotics folks (like yours truly) have some new challenges in the form of barrels added to the path.  There's also a new time bonus of -30 seconds for driving through a 3' arch placed somewhere along the path.

EDIT (1/19/11): Did I miss something? Why yes I did... simultaneous starts in the land category. Five robots run each heat at the same a time. Translation: moving, unpredicatble obstacles! Whee!

All this sounds hard.  Maybe I should build a robot plane instead?

In related news, all the major components for the robot are now in my lab ready to be interfaced.

Moments before I left for the airport, my most recent Sparkfun order arrived, containing a GPS module, sonar range finder, serial LCD, and a couple other fun things.

I've been blessed with good fortune lately. Thanks, Sparkfun, for putting on Free Day, wherein I was able to answer enough questions to subsidize the purchase of some of the above.

Also, thanks to RobotBox for generously awarding me a Trossen Robotics gift certificate for "most active member of the month." My order arrived last week with a long range Sharp IR range finder and a couple other things.

So, I have lots to do when I get back to town.

Can I pull it off?  I don't know. Stick around and we'll both find out.

No doubt it'll be entertaining to watch me once again thrash and panic my way to competition day.

I'll keep you updated on my progress right here!

Interfacing an Arduino with a Rate Gyro

After breadboarding an Arduino for the Sparkfun AVC RC truck robot (somebody help me name this thing) the time was ripe to tackle the LISY300AL rate gyro breakout board I bought from Sparkfun.

My hope is that this device or something like it can be used to help Pokey negotiate 90 degree turns in the firefighting maze and to help the AVC RC Truck bot execute turns and hold a heading.

The breakout board is simple.  There's 3.3V power, GND, analog voltage out, then two logic pins, one for powerdown (PD) and one for self test (ST).  Tie PD and ST to GND, and feed the thing power, then read the output.

I'd previously made a DIY, surface mount (SMT/SMD) 3.3V regulator expressly to power a rate gyro. That gyro is dead now because I accidentally fed it 9V. Oops. I was more careful this time with the new gyro.

I hooked it up to the scope and after realizing my ground line was actually hooked to +5V, I got a proper ~1.65V reading on the scope and on my DMM.

It was time to get an Arduino to read values and spit them out over serial. My breadboard Arduino came in handy, although I wished I'd used a longer breadboard as it's pretty cramped.  I wrote up some simple code to convert analog data and spit it back out over serial.

int gyroPin = 3;
float val;
float sum;

void setup() {
  Serial.begin(38400);
  delay(1000); // allow setup time for gyro
  // try to calibrate to an average
  for (i=0; i < 2048; i++) {
    delay(5);
    sum += (float) analogRead(gyroPin);
  }
  val = sum / 2048;
  sum = 0;
}

void loop() {
  sum += val - (float) analogRead(gyroPin);
  Serial.println(sum, DEC);
  delay(5);
}

The result was ... a mess.  A sine wave output with occasional, negative spikes.  How the heck was I supposed to use this data to steer a car straight?  I played around with capacitors, software averaging, all kinds of stuff, and none of it was a match for this gruesome analog signal.

Then I realized I had left PD and ST floating. I tried grounding them and...

Oh. So that was the problem. *sigh*

Now the graph holds pretty tightly to an average value with little variance.  That's some usable data. The other problem is that Vref for the ADC is 5V and the gyro outputs a max of 3.3V, so I'm losing some resolution.

Calibrating the sensor and dealing with drift are the next problems. I need to find the steady state middle point voltage put out by the sensor.  Then I can read voltage values at a fixed period, convert the voltage value to degrees/sec and integrate (add them up) to find out the current heading. Sounds easy on paper.

After some playing around to get familiar with the device and its output over time, my first experiment with the rate gyro was an attempt to study the noise level from the device while it is sitting still.

I set up simple code to track the change in output between each set of 64 samples (with 2ms delay between each).

I wrote a small perl program to generate a histogram of values and calculate the mean, variance, and standard deviation.

#!/usr/bin/perl

$i = 0;
while (<>) {
  s/[\r\n\s]//g;

  $HIST{$_}++;

  $sum += $_;
  $i++;
}
$count = $i;

$avg = $sum / $i;

$var = 0;
for ($i=-40; $i <= 40; $i++) {
  print $i, ",", $HIST{$i}, "\n";
  $var += $HIST{$i}/$count * ($i - $avg)^2;
}

$stddev = sqrt($var);

print "Average, $avg\nVariance, $var\nStdDev, $stddev\n";

Then I graphed the histogram in the OpenOffice Calc (spreadsheet) and came up with this fairly Gaussian-looking distribution after almost 22,000 samples.

I got a mean of 411.4e-6 (which seems to be drifting a tiny bit, by the way), Var = 162 and StdDev = 12.7280. Aside from being sort of interesting and instructive for simple algorithms, I am hoping that this modeling of the sensor's uncertainty will be useful in developing a Kalman Filter when it comes time to fuse multiple sensors to get more accurate heading estimates.

I'm just starting to dig into the Kalman Filter topic but it seems that some knowledge of the noise statistics of the sensor might be useful, although manufacturers put these in their datasheets. Essentially the Kalman filter is intended to filter out measurement noise to estimate some 'true' value, like heading.

It's a bit involved and I don't pretend to understand it yet, but I gather it's sort of an iterative implementation of least squares curve fitting and is related to Bayesian statistics wherein a new but statistically noisy input is used to refine a prior estimate.

As far as I can tell you take a measurement at each fixed time step, predict where the system will be at the next time step, and use the next measurement to refine the estimate and to make a better prediction for next time step.

All the cool roboticists are doing it.

So, I'm in the middle of reading a number of papers and a book, Fundamentals of Kalman Filtering: A Practical Approach (Zarchan). I hope to know more soon.

Rest assured if I manage to learn anything, I'll share it with you.

Arduino Takes Control (of RC Truck)

Recap: I'm entering the Sparkfun Autonomous Vehicle Competition (AVC) for 2011 (if they have it again), I bought an RC truck for the robot platform.  After playing with the truck a little, it was time to hand the controls over to an Arduino (aka ATmega328P on a breadboard)

This ElectrixRC stadium truck will soon become a robot...

How do you control an RC car with a microcontroller?  It's easy. The steering servo is controlled with servo pulses (surprising, no?) and so is the Electronic Speed Controller (ESC) aka throttle. The Arduino servo library makes coding a breeze.

Image from coastalplanes.com

Wiring is easy too.  The ESC and Servo both use standard 3-wire RC servo connectors.  The ESC has a 5V regulator (Battery Elimination Circuit or BEC). Wire the BEC's 5V and GND to power the Arduino, and wire the signal line to one of the digital pins. Then wire the steering servo's signal wire to another digital pin and connect the power and ground wires so the servo gets power.

After converting my Pololu SPI programmer to serial mode (Pokey and my FTDI programmer were on loan for the Thompson Robotics Expo) I modded some code and... voila:



Here's the code to control the car. Easy peasy. I used digital pins 9 and 10. Getting the range and center set correctly on the servo and throttle took quite a bit of trial and error.  You don't want to force the steering servo to full lock as that is hard on the servo.

// Steering/Throttle test - Michael Shimniok
// Based on Sweep by BARRAGAN  

#include <servo.h> 
 
// create servo object to control a servo 
// a maximum of eight servo objects can be created 
Servo steering;  
Servo throttle;

int ledPin = 13;
int pos = 0;    // variable to store the servo position 

#define CENTER 95
#define LEFT 55
#define RIGHT 130
 
void setup() 
{ 
  steering.attach(9);
  throttle.attach(10);  // attaches the servo on Arduino pin 10 
  
  steering.write(CENTER);
  throttle.write(90); // set mid throttle
} 
 
 
void loop() 
{
  throttle.write(85);
  digitalWrite(ledPin, HIGH);
  delay(250);
  throttle.write(90);
  digitalWrite(ledPin, LOW);
  delay(500);
  digitalWrite(ledPin, HIGH);
  delay(100);
  digitalWrite(ledPin, LOW);
  delay(100);
  sweep(CENTER, RIGHT, 1);
  sweep(RIGHT, CENTER, -1);
  sweep(CENTER, LEFT, -1);
  sweep(LEFT, CENTER, 1);
  digitalWrite(ledPin, HIGH);
  delay(100);
  digitalWrite(ledPin, LOW);
  delay(100);
}  

void sweep(unsigned char start, unsigned char stop, char inc)
{
  unsigned char pos;
  char i;

  for(pos = start; pos+inc != stop; pos += inc)
  { 
    steering.write(pos);             // tell servo to go to position in variable 'pos' 
    delay(15);                       // waits 15ms for the servo to reach the position 
  }
} 

That's one small step of progress with a massive, overwhelming list of things left to do. But hey, every journey, however long and arduous, starts with a single step, right?