Presentation Transcript

1. Building a Reliable Remote Communication Device with Java ME

   JavaOne San Francisco 2013 - CON8004 Leonardo Lima – V2COM (llima@v2com.mobi) Tiago Padovani – V2COM (tpadovani@v2com.mobi)
2. Introduction
3. Aboutthepresenter (Leonardo) Software Development Manager at V2COM, leading the development of both embedded software for edge/field devices and server-side software that receives field data and enables fast communication with remote devices. Eight years of development and architectural experience on mobile, embedded and server platforms to enable management and operation of highly scalable and fast telemetry and network control systems.
4. Aboutthepresenter (Tiago) Lead Software Developer for Embedded at V2COM, leading the development of embedded software for edge/field devices. Six years of development and architectural experience on embedded Java platforms, building reliable communication devices that stays on the field for years.
5. Aboutthepresentation Explore how to build reliable remote communication devices that must stay online, connected, and transmitting field data without interruptions. Using Java ME, a solid foundation was built that can be used to deploy many kinds of applications in the field, with fast, reliable development. In this session, see what the difficulties, ideas, and insights were that led to this foundation, which is currently in use in more than 50,000 devices in Brazil and Latin America to connect more than 800,000 field devices.
6. V2COM is a leading Latin American provider of Smart Grid technologies and Advanced Metering Infrastructure (AMI). Our offer includes hardware, software and services that can reduce losses and increase water and energy efficiency, currently connecting more than 700.000 devices. Besides the economic gains for our customers, we develop technologies with significant impact on the environment (avoiding losses of important natural resources) and with social responsibility (our Smart Grid systems can be used to offer free internet access in challenging areas)
7. Awarded 2013 Java Business Innovation Award JCP member, nominated for EC election More interaction with community to foster Java ME development in this age of IoT!
8. V2COM clients Meter Reading Network Control
9. The scenario
10. IntelligencewareSuite Legacy Systems LAN WAN Terminal Devices IP Network Wireless Network Local Network
11. Use cases Automated Meter Reading (AMR) Read “dumbmeters” andsend data over toUtility Both energyandwatermeters Canbeany meter, really! LikeourZigbierrefrigerator’stemperature sensor, withalarms for too coldor hot! Network control Remotelyconnect SCADA systems tofielddeviceslikereclosures “transparentpipe” between SCADA andDevice DNP3/IEC104 protocolsandproprietary as well
12. Use cases 6 5b 5a 4 3 2 1
13. The techonologyused Gateway: Oracle Java Micro Edition 1 (Java profile IMP-NG version 2 & CLDC 1.1 Hl), embedded in a Gemalto TC65i module. Startedwith Siemens TC45, 6 yearsago! IntelligencewareSuite: Oracle Java Enterprise Edition 5 Oracle Java Standard Edition6 Oracle Utilities Meter Data Management 2.0
14. Living anembeddedlife
15. The 3 commandments “Ifthedevicehas a powerfromsource, it must beon” Ourfirstcommandmentistokeepthedevicealwayson. If it canbeon, it must beon, no matterwhat. “The firmware must beremotelyupgradable” Oneoffewcertainties in software developmentsisthat bugs alwayshappen! So, we must beableto OTAP (over-the-airprovision) thedevice “Ifit’son, it must be online andconnected (ortryingtobe...)” It doesn’tsufficetobeon. It must bealsobeavaiablesothat communication canhappen
16. Hardware choices WatchDog Oneofthefewalways-must-havecircuit for anydeviceis a watchdogtoturn it on in case it turns off orthere’s a lock-up Dual Sim Card GPRS infrastructureisnotveryreliable, so it shouldbeabletohaveoption in case ofcarrierfailure Dual Serial Manyfielddeviceshavetwo serial ports, sobeingabletoconnecttobothatthesame time isveryimportant Java ME Beingabletorun Java ME allowsmanyadvantages over C
17. Software choices - Separationof concerns “Hardcore” Abstracts boardspecificsdetails (like GPIO) “Zion” Handles common infrastructure (backend connection, messagequeues, locksandsoon) Abstracts Modem commands, sothere’sone for eachavaiable module (Cinterion/Gemalto; Motorola; Telit) “App” Holdsapplicationlogic - As tailored as clientsneed it tobe Ranges from a simpleTransparent (TN) toAutomated Meter Reading (AMR)
18. Hardcore Handlesboardspecifics Modem/AT Connection SMS/GPRS logic GPIO Watchdog
19. Zion Defines commandshell Handle TCP communications Kitchensinkutils (math, string, locks, queue) Event Bus Extendabletohandle “fatter” functionalities CSD SNMP monitoring
20. Application Handlesonly business logicandprotocol Fastchangestoacommodate business logic Afterthefirstfewcicles, it stabilizes
21. PackageDiagram mobi.v2com.zion... Application Control Hardcore Communication Event Protocol Tools Utilities ... andapplicationspecifics as well
22. Challenges Problemsfaced, solutionsprovided!
23. Howtokeepthedeviceon? Problem: Module canthrowExceptions (EXIT CODEs) thatshutsdowntheapplication Solution: Hardware Watchdog! Dedicated QG8 tohandleonlythis (poweron/off) 10 minutes timeout: eitherthe firmware “swings” the GPIO, or it getspowered off thenon Manyother software watchdogs... Manyotherconditionscanmean a “lockup”
24. Howtokeepthedevice online? Problem: GPRS connections canbeunstable, speciallywhenneededmost: thunderstormsandharshweathercondition Solution: Dual-SIM cards, soif a network fails, therecanbeanother Many soft watchdogstoretryattachments Turnon/off modem, likeourphones Attach/Deattach
25. Howtogetbehindpublic APN firewall? Problem: Can’t use a server socket whilecommunicatingonpublic networks! Solutions: Devicepolls server, asking for new commands Thisdoesn’twork for always-on connection needs Expensiveonresources Software Switch (server side)! Meet-in-the-middle approach
26. Howtokeepthe TCP connection up? Problem: Youdon’tknowthe connection isbrokenuntilyouhave a blockingread() orwrite() TCP keep-aliveisnotenough No module supportfor this... Solution: Keep-Alivepackets Application-level bytes senttomakesure connection isnot off
27. Howtokeepthe firmware updated? Problem: There are MANY devicesdeployed, eachwith a software version No onewantstomass-deploy... Notallupdates are required for alldevices Solution: Axonhascontrolofeachversion Updates are dispatched as neededand/orclearedbyclient
28. Howtodebug in-device? Problem: No simplelogging framework towrite log to Log redirectionworks for serial portonly, notremote connections Solution: Built a simplelogging framework thathandlesdebug level, message, threads
29. Howtodebug off-device? Problem: Howcanwerun/testoutsidethe framework? Many classes youdependon are hardware specific (Files, Connections) Solution: Built a simple “javaabstractionlayer” thatimplementsjavax.microedition classes (andproprietaryones too) that runs on Java SE onthe desktop.
30. Howtokeephistorical, debugginginfo? Problem: No long-termstorageondevicestokeep logs Flash memorycanholdatmost 30 minutes of logs The logs youneed are theoneswhendeviceisunreachable, but alas, youcan’t download it. Solution: Have a “black box” flight recorder Small, coded files thatcanholdcriticalinformation for weeks
31. Howtosynchronizeaccesstoresources? Problem: Access to system resources must besynchronized Serial device, system files synchronizedcan lead tounknownlockups Some raceconditions hard totrackando also some bugs... Solution: Customimplementationof Java SE’sLock ReentrantLockinspired Addedfewmethods for convenience tryLock, sothatyoulockifyoucan, butdon’tfailifyoucan’t fastLock, for locksthatshouldlastthan 2 secods
32. Howtodefault parameters? Problem: Same firmware for differentclientswithdifferentneeds, lotsofparameters Parameters for APN, Serial, Timings andsuch Flash memorycangetcorruptandyouloseyourparameter files! Solution: Use JAD parameters JAD canbecustomtailoredtoeachclient Canprovidegood defaults
33. Pendingchallenges... ... becauseyoucan’twinthemall...
34. Manypain points are still there Makesmallerbinaryfragments? Andsmallupdates over theair. Runmultipleapplicationsatthesame time? More “permanent” and/orremotelogging Security issues Better management ofkeys Anti-tamperingmeasures
35. Incomingchanges Java ME (r)evolution
36. Java ME 8 hasmuch new features! Java ME 8 seemstohaveaddressed some, ifnotmost, thisproblemsandabstractions Device I/O Access Point NIO ... Andmanyothers! Some still needworks No concurrencysupportmightmean no locks... We’llworkwithJSRstotryandaddresstherestofthem V2COM isjoiningthe JCP EC to help foster Java ME evolution
37. Q?
38. Thanks! llima@v2com.mobi www.v2com.mobi