Creating an H.264 stream from Unity Content

Argus Simulator

Currently I’m working on a project at work that requires the information (lets call it unity content) displayed on twelve screens in another room/building (a long way away from the computer generating it), it’s a little bit of an unrealistic-solution/challenge to connect a dvi/hdmi/<or you name it> cable through buildings… as quality degrades the longer the cable gets… and not to mention all other obstacles along the way.

Enter streaming over ip… now this is not the easiest thing to do there are many ways to go about it.

The first option that came to mind was lets get a hardware adapter that can convert the DVI output to an H.264 IP stream.. that is a pretty expensive solution if you have twelve outputs that need to go to twelve screens, average cost of a decent hardware encoder runs somewhere around and above 4000 euros that times 12 and you have yourself an expensive streaming setup..

So Ideally thinking about costs the best solution would be do it in software, the solution first thought of was to do screen capture and encoding on the fly, all possible with vlc and many other software screen capture programs. After a few small tests this proved to be really CPU hungry and not really the best solution, because we need as much CPU & GPU time as possible for generating the Unity Content.

So now almost at the end of all options a colleague of mine mentioned AMD’s Radeon Sky Series which is basically cloud gaming tech not yet available for the masses. This got me thinking and looking more closer at cloud gaming which is basically a server on the web streaming games to specialized clients. Looking further into this I found Nvidia and the new range of GPU’s with Kepler Architecture *hallelujah* each Kepler based card is fitted with a beautiful h.264 encoding chip. Which turns out to do encoding without even touching the GPU or CPU time, and mighty fast too… 76 frames at 600ms which boils down to about 8ms per frame  which is what i’m looking for. with Low Latency settings I even got a 1.08ms encoding time… *jaw drop*

So I dove head first into the reasonably clear Nvidia API and expanded on my previous experience in creating a decent C++ native Unity Plugin. After bumping into alot of “unresolved external symbol” errors (which by the way usually mean your missing a lib file) and a few unicode/ascii hurdles (Yes i didn’t think this would be an issue, but it turns out C++ is picky about string encoding) I eventually got some output from the chip even though it was just a green screen.. it was still output and the encoder had been initialized to create it. Which meant progress!

Realizing that the green screen wasn’t my actual unity content I figured I’d better dig a little deeper and found that the green screen was actually blank YUV pixels, eventually I managed to change that to red using different YUV values and filling the texture with them. Still not so useful because Unity doesn’t output YUV pixels now does it. This led me to search the internet for a way to convert RGB to YUV (actually BGRA to YUV444) pixel for pixel conversion was an option with readily available formulas online.. only not the fastest way.
Browsing through the built-in shaders I found a reference to a YUV conversion, using that as a reference I created a new shader for myself to do the YUV conversion with some modifications to values and the order they are stored in.

Now a little under half a year later i’m using my plugin to encode 12 H.264 streams to send to VLC which passes it on to a VMS.
Video Will soon follow 😉

Arduino: kWh Monitoring

Inspired by a fellow ne2000-event goer (dutch site) who made a kwh-meter-meter. I decided to also do something similar with my Arduino as it was just sitting idle in a box waiting for greatness…

To do such a project requires a light amount of electronics experience, although you can probably do it without. If you know how to solder and you can tell the difference between positive and negative connections then bob’s your uncle.

Before I could get started on this project I needed a few more electrical components like resistors, more flashy leds and what not. I ended up searching the web for a decent place to order the components (which turned out to be and ordered the IR Sensor for the project from a local electronics shop in Utrecht because didn’t have it. Three days after ordering all the bits and pieces a package arrived and I could start.

I soldered the TCRT5000 sensor to a piece of soldering board because I didn’t want to break any of its terminals fiddling with cables and such, plus it made for a better connection. Added the resistors and two leads for power to that piece of soldering board and then connected it up to the arduino read out the value and.. awesome it worked straight away.

After some tweaking of values I got a sensible reading from the Arduino for each cycle on my kWh meter. That was step one actually getting an event outside of the kwh-meter. Now the question was what to do with that.

In the example project that inspired me (juerd’s project) juerd had an OpenWRT router capturing values and storing them into an rrd file. As I dont have any OpenWRT or linux server I thought why not use Munin to monitor my Arduino output.

To accomplish this I used my Ethernet shield as a webserver to serve values to a custom made munin plugin (which was easy to make). That saved me a problem because I already had munin running on a server of mine.

In using munin I have a polling time of 5 minutes which is more than enough to see the amount of power going out, but it would mean that if something used an excessive amount of power in a time period of less than 5 minutes then I could potentially miss it. To fix this I added a variable that gets reset after each web request and holds the highest value: maxwatt. That allows me to see when short bursts occur in my power usage. But overall the 5min poll time is just fine.

Yes it’s fairly low-tech at the moment but hey come on.. it’s an arduino project. Project code can be found here

A Quick update that this project is still working..
It needed a slight adjustment because my provider hands out dynamic IP addresses, and the arduino lost track of its self after a while.
So I had to change the order of execution in the loop and also added some dyndns updating..
Here’s the latest day graph: