Continuous Integration Server in a box

So at work, we needed a simple continuous integration server for testing purposes. The lab I work in is closed off from internet access and I don't have admin privileges, so I needed a solution that would work with no help from IT. Using git as my configuration management tool, here is what I ended up using, rather nicely as it turns out.

The product is an open-sourced tool written in node.js called Concrete [1]. The setup is basically this:

On a machine with internet access,

1. Download and drop on the computer node.exe [2]. As an example, use c:\Users\keith\tmp\node

2. Download and expand npm archive [3] in the same directory as where node.exe resides.

3. Open a command prompt and go to the node.exe directory Run npm install concrete

  C:\Users\keith\tmp\node\npm install concrete

4. (Optional) Install node's IRC module. The node IRC module will be used to modify Concrete to add IRC notification and demonstrate how easy it is to hack this simple tool.

  C:\Users\keith\tmp\node\npm install irc

5. Get unxutils [4] or gow [5] or some form of curl.exe. Curl is used via git-hooks to signal the continuous integration server that a change has been pushed up to the repository.

6. Zip up the node/npm/npm_modules stuff

7. Download mongodb [6]

8. (Optional) Get bewareircd [7] and irrsi [8], an IRC server and client.

9. Transfer the node stuff and mongodb archives as well as curl and the optional downloads to a machine on the target network.

Now on your desired test machine.

1. Unzip the node stuff. I will use C:\tools\concrete_in_a_box\node as an example

  C:\tools\concrete_in_a_box\node\node.exe
  C:\tools\concrete_in_a_box\node\npm.cmd
  C:\tools\concrete_in_a_box\node\node_modules\
  etc.

2. Unzip mongodb. Again, I will use C:\tools\concrete_in_a_box as an example

  C:\tools\concrete_in_a_box\mongo

3. Open a command prompt and change paths to the mongo directory. Start mongo, making sure that the invocation states where the database will reside. The default is C:\data\db, but I prefer to keep everything self-contained. So I make a directory inside the concrete_in_a_box directory called mongo_data.

  C:\tools\concrete_in_a_box\mongo\mongod.exe --dbpath ..\mongo_data

4. Open another command prompt. Clone the git repository of interest. For this simple case, let's assume that there is repository of interest in C:\repos\my_project and we clone it in the concrete_in_a_box directory

  C:\tools\concrete_in_a_box> git clone c:\repos\my_project 

5. Add a concrete job runner. As an example, assume that there is a build script batch file named build.bat as part of the project

  C:\tools\concrete_in_a_box\my_project> git config --add concrete.runner="build.bat"

build.bat might be something as simple as invoking msbuild to build a Visual Studio solution. As an example,

@REM build.bat
@echo off
@REM make sure that msbuild is available (assuming VS2012)
@if "%VSINSTALLDIR%"=="" call "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\vsvars.bat"
msbuild my_project.sln /p:Configuration=Debug;VisualStudioVersion=11.0
@exit \b %ERRORLEVEL%

6. Use hooks to signal to Concrete that changes have occurred. In this particular example, we modify the .git/hooks/post-receive hook of the repository we cloned for Concrete to invoke curl to post just blank data to the IP and port that Concrete will be running on.

  C:\repos\my_project\.git\hooks> <edit post-receive>

#!/bin/sh
#
# An example hook script for the "post-receive" event.
#
# The "post-receive" script is run after receive-pack has accepted a pack
# and the repository has been updated.  It is passed arguments in through
# stdin in the form
#  <oldrev> <newrev> <refname>
# For example:
#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
#
curl localhost:4567 -d Build

7. Open another command prompt. Add to the execution path the node.exe directory and the node_modules\.bin directory

  C:\> set path=%PATH%;C:\tools\concrete_in_a_box\node;C:\tools\concrete_in_a_box\node\node_modules\.bin

and then execute concrete

  C:\tools\concrete_in_a_box\my_project> concrete .

8. Open your web browser and observe Concrete sugary goodness at http://localhost:4567.
Now, anytime that a change is commited and pushed to the repository at c:\repos\my_project, it triggers the post-receive hook which issues a POST command to the Concrete webserver, which invokes the runner. In this particular case, it pulls down the latest from its git origin (c:\repos\my_project) and then runs build.bat.

9. If you have not created a build-worked or build-failed hook file in your .git\hooks directory, it will bark at you saying it cannot find it. I simply created little bat files that echo whether the build succeeded or failed. One thing I did was I modified the Concrete git.js library file and explicitly called the files it was looking for to be build-worked.bat and build-failed.bat.

diff git.js original_git.js

23,24c23,24
<       git.failure = path.normalize(target + '/.git/hooks/build-failed.bat');
<       git.success = path.normalize(target + '/.git/hooks/build-worked.bat');
---
>       git.failure = path.normalize(target + '/.git/hooks/build-failed');
>       git.success = path.normalize(target + '/.git/hooks/build-worked'); 

Optional fun!
So we also included some IRC stuff; the lab we have does not have an email server set up and I wanted to have instantaneous feedback on when a build occurred and what the outcome was. I figured it would be very simple to plumb in a little IRC bot, and using some example code from here [9], I was able to get one up and  running in about 15 minutes, including the setting up of the IRC server and client. The only modification I did was to runner.js

diff runner.js original_runner.js

10,25d9
<
<   // Create the configuration
<   var ircConfig = {
<       channels: ["#concrete" ],
<       server: "my.ownircserver.net",
<       botName: "concretebot"
<   };
<   // Get the lib
<   var irc = require("irc");
<
<   // Create the bot name
<   var ircBot = new irc.Client(ircConfig.server, ircConfig.botName, {
<       channels: ircConfig.channels
<   });
<
<
114d97
<                 ircBot.say(channel, "Build failed!");
121d103
<                 ircBot.say(channel, "Build succeeded!"); 

Unzip the bewareircd archive and modify ircd.conf so that the server name matches the server name in runner.js (e.g., my.ownircserver.net). Start 'er up.

Unzip the irrsi archive and launch irrsi. Add the server (in this case localhost), connect to it and join the #concrete channel. You'll see that there is a concretebot in the room; everytime that runner is invoked and reaches either success or failure, the concretebot will say the room that particular run status. This can easily be expanded to spit out a ton of other information such as which build and committer started the runner. Handy enough if you don't have time to hit refresh on a web browser.

[1] https://github.com/ryankee/concrete
[2] http://nodejs.org/download/
[3] http://nodejs.org/dist/npm/
[4] http://sourceforge.net/projects/unxutils/
[5] https://github.com/bmatzelle/gow/wiki
[6] http://www.mongodb.org/downloads
[7] http://ircd.bircd.org/
[8] http://irssi.org/
[9] http://davidwalsh.name/nodejs-irc

Homebrew Version of Steve Webber's 'Tenna Dipper

Since I still had the wild hair for soldering, I rummaged through my junk box and realized I had most of what I needed to build a very simplistic antenna frequency dipper, to be used as a poor man's antenna analyzer. The design is a Steve Webber original [1], and has been kitted by several groups before. The circuit seemed simple enough that I could try and build it up on some perfboard. I ordered a couple of chips I didn't have from Tayda Electronics [2], and going off of this schematic[3], went nuts. I think I have it wired up correctly, I'm getting proper ring-outs on Vcc points and ground, and the output to the antenna/atu BNC is a nice 50 ohms. As I don't have an antenna that I know its resonant frequency, I'm at a bit of a loss to actually test it. Oh well...its day will come. Here it is in all its glory: my homebrew version of the 'Tenna Dipper.

[1] - http://kd1jv.qrpradio.com/tennadipper/tenna_dipper.HTM
[2] - http://taydaelectronics.com
[3] - http://kd1jv.qrpradio.com/tennadipper/atrlb.gif

Tayloe SWR indicator

I felt the urge to do some soldering, so I went looking for a simple project. I thought that having an SWR indicator would be nice to have, as I have all the requisite components on hand (including some high power/high precision resistors).  What this little contraption does is basically give a visual indicator (no-lit or minimally lit LED) when the SWR is as closely matched to 1:1 as possible. The idea is hopefully pair this with a magnetic loop antenna and that way I won't have to bother with an antenna tuner. For more information on the Tayloe SWR indicator, please check out qrpkits.com [1].

Here is my uber-high quality work:

It rang out correctly, with open circuits on the transceiver side and antenna side with the circuit removed, and 75 ohms and 50 ohms when the SWR circuit is inline.

---

[1] - http://www.qrpkits.com/swrind_case.html

Blank GoodFET board

I just received Travis Goodspeed's GoodFET board [1] in the mail this week. I have ordered the requisite parts to populate it, have practiced some drag soldering for the SMT parts, and hopefully later on this week I can assemble it. Let's see how bad I butcher it...

Update: 
Pretty bad, as it turns out. I was sloppy with my solder braid and accidentally lifted several pads on the FTDI chip. However, I did place the MSP430 chip nicely in place. So a quasi-expensive lesson, but one that I definitely learned from. I feel pretty confident that I can handle TSSOP scaled SMT work now, even if it may not be pretty.

---
[1] - http://goodfet.sourceforge.net/

Fun with an MSP430 Launchpad and a Digilent Nexys3 FPGA demo board

At my office we had an internal class for studying VHDL, using Digilent Nexys3 as our experimentation board  [1]. As an end of class project, my partner and I opted to try and create a simple digital theremin.

The plan was to take an input from an IR sensor (like the Sharp IR proximity sensor [2]) and feed it through an ADC to generate a digital word. This word could then be manipulated to generate a frequency tuning word used for direct digital synthesis [3]. This would generate an output digital word corresponding to a variable frequency sine wave, which was then fed through a DAC and feed into a speaker for output.

The Nexys3 has a Xilinx chip, and we used the ISE Webpack development tools, which gave us the option of using Xilinx's DDS IP core. However, as part of our learning experience, we opted to use an open source core and incorporate that into our design. Martin Kumm developed a handy DDS core that we found on opencores.org [4], and this was the starting point for our project. Martin kindly provided a working test bench that we could simulate and understand the outputs of the core. Since we were only generating a sinewave to drive a speaker, we weren't interested in modifying input phase or capturing output phase.

As the Nexys3 board provided some useful peripherals, we adapted our original design to implement useful features, including an ability to set a specific frequency based upon the bank of switches, as well as displaying the generated frequency on the seven segment display.  As for limitations, since we were interested in simple testing, we fixed our output to be capped at 20kHz.

As seen in the following picture, we used the PMOD adaptors directly as pin inputs, with the JA bank used for the DDS output and the JB bank used for the digital word input. Originally we were going to use an ADC PMOD, but since it used an I2C interface we opted to go for a simpler direct GPIO approach. We used the seven rightmost switches to act as a seven-bit digital word, with each bit representing 190 Hz to give us our spread in spectrum. The final switch acted as a relay, using either the switch bank or the seven LSB pins of JB as the source for generating the frequency tuning word.

The following block diagram, while not 100% exactly correct, provides a coarse overview of how top-level design was mapped out. The dds_synthesizer is the DDS core we found on opencores.org. The clkdiv is a simple clock divider to provide useful sub-interval clocks from the master 100 MHz clock.

Our DAC was a simple 8 bit R2R laddder [5] soldered on a protoboard, again for the simplicity of not trying to have to use an intermediate communication protocol like I2C or UART. As shown in the image below, you can actually observe the discrete steps of the ladder as each bit became active on word transitions.

The addhalf block shown in the top-level design shifted the output of the dds_synthesizer since we were expecting a unsigned output for a DAC functionality, and it pushed out a signed word. That was a particularly fun thing to figure out; our output basically was a chopped sinewave with the peak and base at median value (as shown below), with discrete jumps at the transition boundaries

Once we had the switch bank working and the output generating a tone, we moved to the ADC side. Ruling out the PMODAD* series from Digilent, we elected to use a simple microprocessor's ADC and push out the digital word on pins configured for GPIO. As this was a really quick hack, I grabbed an MSP430 Launchpad, used the Energia IDE and read an analog signal on one pin and pushed it out digitally on 7 pins. Its wiring diagram is shown below.

Lo and behold, it worked quite well; the audio was pretty weak on the reclaimed PC speaker I stuck in there, but it was very effective.

The source code for the FPGA and the microprocessor can be found here [6].

There were some other cool projects that were ginned up by the class, including an IR edge detector via the ADC PMOD thrown on top of a simple two wheel autonomous vehicle, a spoofing of the WWVB clock setting signal, the groundwork for a couple of different games including a simple tower defense game as well as a version of Simon, and some integration of accelerometers (for the makings of a simple IMU) among others.

---
[1] - http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,897&Prod=NEXYS3
[2] - https://www.sparkfun.com/products/242
[3] - http://en.wikipedia.org/wiki/Direct_digital_synthesizer
[4] - http://opencores.org/project,dds_synthesizer
[5] - http://en.wikipedia.org/wiki/Resistor_ladder
[6] - https://github.com/kpimmel/digital_theremin_fpga

Flashing the firmware on a Nokia Lumia 810

I recently received a new phone, but unfortunately it got into a bit of a soft bricked mode, where the phone would never progress past the Nokia boot up screen. Following the instructions found here on wpcentral.com [1], with modifications reflecting the use of the 810 instead of the 920, I was happily using a newly flashed phone operating perfectly.

---
[1] - http://www.wpcentral.com/how-fix-bricked-920-after-reset

Stellaris, Cygwin and OpenOCD for Debugging

So in my previous post, Chris laid down the challenge in the comments to set up debugging within Cygwin [1]. As I don't know anything about it, I figured I'm the perfect fool match to do this task. Chris kindly provided me  a link as a start [2], and from there I proceeded to let google be my friend to supplement that background. This ended up yielding a nice tutorial from Mauro Scomparin [3] via Hack-a-Day[4]. With some timely help from Spencer Oliver [5], OpenOCD developer-extraordinaire, I now have a means of debugging the board under program. So know you know the rabbit trail. So what are the basic steps to get debugging under cygwin?

1. Make sure you have appropriate tools and libraries installed for Cygwin
2. Get the latest code base of OpenOCD
3. Get the windows version of libusb set up in cygwin
4. Build the sucker
5. Play gently with the sucker

Step 1. Make sure you have the appropriate tools.
Let the errors be your guide. I wish I could tell you specifically what packages you need, but at this point, what I've installed and what is actually necessary are not easily separated. One thing to note: please make sure the package tcl (under 'Interpretors') is *not* installed; openocd includes an embedded tcl engine called jimtcl, and for whatever reason, it really does not like the cygwin tcl package existing with its nasty DLLs and whatnot. If you're going to use the exact same commands as I have listed below, make sure to install the mingw-gcc development tools as well.

Step 2. Get the latest code base of OpenOCD

It is easiest to do this with git. From your cygwin prompt

$ git clone http://git.code.sf.net/p/openocd/code openocd

Step 3.  Get the windows version of libusb set up in cygwin

We need to get the libusbx library and headers[6] if we're going to cross-compile with the mingw cross-compiler in cygwin.  Add the header and lib files to the compilers. For 32-bit sugary-goodness, create /usr/i686-w64-mingw32/sys-root/mingw/include/libusb-1.0/ and place libusb.h into it (note call it libusb-1.0 not libusbx-1.0. For 64-bit magic, create /usr/x86_64-w64-mingw32/sys-root/mingw/include/libusb-1.0/ .

For the record, I used the static library and not the shared object/DLL version.

Step 4.  Build the sucker

Using Spen's original guidance, I decided to just cross-build it using the 64-bit mingw packages from cygwin.

$ cd openocd
$ ./bootstrap
$ ./configure --enable-maintainer-mode --build=i686-pc-cygwin --host=x86_64-w64-mingw32 --disable-shared --disable-werror --enable-stlink
$ make

If you wanted to build it with 32-bit mingw cross-compilers, then you would use instead

$ cd openocd
$ ./bootstrap
$ ./configure --enable-maintainer-mode --build=i686-pc-cygwin --host=i686-w64-mingw32 --disable-shared --disable-werror --enable-stlink
$ make

Trying to build it natively at this point under cygwin didn't work for me, but I was able to run with the mingw-built binaries, so that's good enough for government work and me.

A quick note here; the --enable-stlink flag, as Spen notes, builds for the ti-icdi as well, as they use the same driver backend.


5. Play gently with the sucker

Pretty easy up to this point. Like Mauro, we're not going to invoke make install, but rather I chose to dump the requisite executables and support files into a separate directory. In my case, that is /home/pimmel/work/stellaris/openocd. Basically copy the contents of the tcl directory and openocd.exe executable to where you want. Following Mauro's guide, we need a configuration file for OpenOCD.
Use the following text and save it as LM4F120XL.cfg in the openocd-bin folder  (again from Mauro):

# TI Stellaris Launchpad ek-lm4f120xl Evaluation Kits
#
# http://www.ti.com/tool/ek-lm4f120xl
#
#
# NOTE: using the bundled ICDI interface is optional!
# This interface is not ftdi based as previous board were
#
source [find interface/ti-icdi.cfg]
set WORKAREASIZE 0x4000
set CHIPNAME lm4f120h5qr
source [find target/stellaris_icdi.cfg]

Now we’re ready to launch it; connect the board to a USB port and just type in the command line:

./openocd.exe --file ./LM4F120XL.cfg

Next connect arm-none-eabi-gdb to OpenOCD and load the program! First start the debugger with something like this:

arm-none-eabi-gdb main.axf

This will open up gdb for arm-none-eabi with a target file called main.axf. Next, at the gdb prompt connect to the OpenOCD interface via the following commands:

target extended-remote :3333
monitor reset halt
load
monitor reset init

Now the target processor is halted at the beginning of the startup_handler() code of the LM4F_startup.c file. At this point, just use the standard gdb commands to set breakpoints and display variables. From Mauro's tutorial here are some examples:

b main.c:51
b Timer1A_ISR
display count
c

A tutorial on gdb is beyond the scope of this little write-up, but it is an intuitive and easy program to use. So now we have another tool in our box to help our development along.
Inconclusion?
So now we have a working open circuit debugging interface, which upon after launching, we can attach a working program and control it via gdb. Pretty cool...now time to dust off my gdb reference material. Much thanks to Spen and Mauro on doing all of the heavy lifting in getting this stuff put together.

[1] - http://underwaterwhistler.blogspot.com/2013/01/getting-started-with-stellaris-launchpad.html
[2] -  http://elinux.org/Compiling_OpenOCD_Win7
[3] - http://scompoprojects.wordpress.com/2012/11/07/debugging-a-program-on-the-stellaris-launchpad-board/
[4] - http://hackaday.com/2012/11/19/how-to-build-openocd-with-stellaris-launchpad-support/
[5] - http://forum.stellarisiti.com/topic/490-openocd-and-cygwin-causing-stackdump/?p=2409
[6] - http://sourceforge.net/projects/libusbx/files/releases/1.0.14/Windows/libusbx-1.0.14-win.7z/download