{"id":2476,"date":"2013-08-30T09:31:23","date_gmt":"2013-08-30T07:31:23","guid":{"rendered":"http:\/\/svay.com\/blog\/?p=2476"},"modified":"2013-10-21T22:13:15","modified_gmt":"2013-10-21T20:13:15","slug":"connecting-mobile-html5-to-hardware","status":"publish","type":"post","link":"https:\/\/svay.com\/blog\/connecting-mobile-html5-to-hardware\/","title":{"rendered":"Connecting mobile HTML5 to hardware"},"content":{"rendered":"<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<p><iframe loading=\"lazy\" title=\"Arduino device controlled by an HTML5 app\" width=\"580\" height=\"326\" src=\"https:\/\/www.youtube.com\/embed\/iaTM_yRtCx4?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>Initially, this article was supposed to be &#8220;How to add an external flash to a Firefox OS phone&#8221; but my project did not work out. The idea was to add an external light to the camera of the Geeksphone Keon, as it doesn&#8217;t have any built in. Unfortunately, I couldn&#8217;t complete the project, here is why.<\/p>\n<p><!--more--><\/p>\n<h2><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> The plan<\/h2>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> My plan was to build a simple photo app that could trigger an external light via the headphone jack. All smartphones have this socket and usually do not require &#8220;hardware certification&#8221; to work. <a href=\"https:\/\/squareup.com\">Square<\/a> is doing something similar with their card reader, which is pretty cool.<\/p>\n<h2><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> The setup<\/h2>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>The hardware part of the project was fairly easy. All you need is:<\/div>\n<div>\n<ul>\n<li>an Arduino<\/li>\n<li>a soft modem<\/li>\n<li>some ultra bright LEDs and resistors<\/li>\n<li>a battery<\/li>\n<\/ul>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>Communicating with a phone through the headphone socket doesn&#8217;t require a lot of electronics.\u00a0The web has <a href=\"http:\/\/www.flickr.com\/photos\/arms22\/4821347734\/\">schematics<\/a> so you can build a soft modem with simple components. Another option is to buy\u00a0one on eBay, <a href=\"https:\/\/www.sparkfun.com\/products\/10331\">Sparkfun<\/a> or <a href=\"http:\/\/dx.com\/p\/softmodem-audio-modem-module-iphone-and-android-223833\">DX<\/a>, ready to use.<\/div>\n<div><\/div>\n<div><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> These kinds of modems use <a href=\"http:\/\/en.wikipedia.org\/wiki\/Frequency-shift_keying\">FSK modulation<\/a> for the communication. Like good old modems, they use audible tones. If you plug speaker you can hear very fast blip sequences (think R2D2).<\/div>\n<div><\/div>\n<div><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> There is a\u00a0<a href=\"http:\/\/code.google.com\/p\/arms22\/\">SoftModem library<\/a> that does the work of generating and reading FSK signals on the Arduino. It only works with Arduino Uno (as it needs specific timers) and you will need a <a href=\"http:\/\/code.google.com\/p\/arms22\/issues\/detail?id=2\">patched version<\/a> to be able to compile it with Arduino 1.0.<\/div>\n<div><\/div>\n<div><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> On the browser side, there is a <a href=\"https:\/\/github.com\/NeoCat\/FSK-Serial-Generator-in-JavaScript\">JavaScript library to generate an FSK signal<\/a>.<\/div>\n<div><\/div>\n<div>So far so good?<\/div>\n<h2><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> Hardware was easy, HTML5 support sucks<\/h2>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> Unexpectedly, building the software was the most difficult part of the project.<\/p>\n<h3><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> 1. The Keon doesn&#8217;t redirect sound to the headphone socket<\/h3>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>The project will probably not work with the Keon as initially planned. It appears that the device will only redirect sound to the headphone socket if you plug an actual headphone in it. Plugging a TRRS cable only won&#8217;t work and sound will come out from the internal speaker instead.<\/div>\n<div>The iPhone supports this feature, so why not try on the iPhone?<\/div>\n<div><\/div>\n<h3><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> 2. Safari on iOS can not play sounds in dataURI<\/h3>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> Even if the iPhone redirect sound to the headphone socket, it can not play the generated FSK signal that is rendered as a Base64 dataURI. It just doesn&#8217;t.<\/p>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>My solution is to pre-record the signal and play it as a wav file (with proper MIME type, which is important).\u00a0You could also use more <a href=\"http:\/\/html5doctor.com\/taking-web-audio-offline-in-ios-6-safari\/\">advanced stuff from the Web Audio API<\/a>. But in my case, I only have a few commands, so pre-recording is OK.<\/div>\n<div><\/div>\n<div><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> At this point, I can send commands from the phone to the external flash. Let&#8217;s build the camera app.<\/div>\n<h3><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> 3. WebRTC support is very poor<\/h3>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>My idea was to use getUserMedia to get the live stream from the camera. Unfortunately, <a href=\"http:\/\/caniuse.com\/#feat=stream\">getUserMedia isn&#8217;t supported by any mobile browser<\/a>\u00a0at the moment.\u00a0It works on Firefox for Mac, but not on Firefox OS. I works on Safari for Mac, but not on Safari for iOS.\u00a0Even Phonegap doesn&#8217;t help on the iPhone.<\/div>\n<div>The closest thing would be a file input, but taking a picture usually happens outside of the browser.<\/div>\n<div>No getUserMedia, no photo app.<\/div>\n<h2><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--> Conclusion<\/h2>\n<p><!--?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?--><\/p>\n<div>At the end, the HTML5 camera app with an external flash only works well in desktop browsers. On mobile, all I can do is trigger the flash, but not take a picture. Pointless isn&#8217;t it?<\/div>\n<div>Anyway, communicating between from an HTML5 app to a third-party device works and we can probably build more cool things with it.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Initially, this article was supposed to be &#8220;How to add an external flash to a Firefox OS phone&#8221; but my project did not work out. The idea was to add an external light to the camera of the Geeksphone Keon, as it doesn&#8217;t have any built in. Unfortunately, I couldn&#8217;t complete the project, here is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[29,26,27,21],"tags":[45,46],"_links":{"self":[{"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/posts\/2476"}],"collection":[{"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/comments?post=2476"}],"version-history":[{"count":9,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/posts\/2476\/revisions"}],"predecessor-version":[{"id":2488,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/posts\/2476\/revisions\/2488"}],"wp:attachment":[{"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/media?parent=2476"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/categories?post=2476"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/svay.com\/blog\/wp-json\/wp\/v2\/tags?post=2476"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}