Commercial grade IoT solution rebuilt in record time
How we rebuilt a commercial grade IoT solution in record time (after an offshore vendor failed to deliver)
Hardware, Firmware, IoT, Cloud
RaspberryPi, IoT Gateway, AWS IoT
From reimplementation to go live
Project Overview
A large-scale industrial IoT Cloud solution for the Transportation & Logistics industry
Hardware:
- Automated / smart control of electrical and gas based heating systems.
- Sensors for measuring environment such as temperature, gas pressure, etc.
- Actuators (electromechanical, and electrical) to control gas flow, temperature.
- Control circuits
- Safety controls
- LCD displays allowing local operators to control settings.
- Deployed in remote locations.
- Fleet of 200+ devices.
Firmware:
- The firmware implements various control algorithms (such as proportional–integral–derivative controller PID)
- Reading data from sensors
- Driving actuators
- Display communication
- Cloud communications (AWS IoT)
Cloud:
- A cloud based control panel - web application - is used to control and monitor systems operations
- Monitoring and Alerting capabilities (email, sms alerts)
Our client wanted to implement a robust, fault-tolerant, industrial grade IoT solution. To do so he contracted what should have been a reputable offshore vendor to do the hardware circuits design, as well as implement the firmware, and cloud platform.
Problems with offshore vendor:
- A year and a half, and hundreds of thousands of dollars later, development was still ongoing with no end in site.
- Client's testing indicated a large number of critical defects that kept piling up with no resolution.
- The project delivery falling behind. The system was not operational.
- To make matters worse our client had contracts in place that were now at risk with penalties involved.
Execution
When the client engaged us, he didn't have visibility into all the underlying issues. He only saw the symptoms, but the root causes were not clear.
Assessment:
We first engaged the existing vendor delivery team to understand why the project is falling behind. A few days were spent engaging the vendor resources, and one of the main issues became clear. The team didn't have the required seniority and experience to deliver on a large scale IoT project.
Project audit:
Our second step was to conduct a full project audit, reviewing the hardware designs, schematics, final PCBs, and the software deliverables involved. Upon inspection it became clear that critical technical decisions were differed to junior team members. The client was provided with an audit report that highlighted critical issues.
- Hardware design flaws were identified (ex. poor noise filtering)
- Firmware design flaws were identified. Ex. firmware was implemented using 3 different technologies, with multiple processes, with poor inter-process communication.
- Architectural and implementation flaws were idenfitied. The vendor went down the path of implementing their own in-house fleet management and cloud communication, instead of relying on industry standards such as MQTT, and using AWS IoT Cloud for Fleet Management and Communication capabilities.
Remediation Plan:
- Our last step was to propose a plan of action. We clearly identified which parts of the system and implementation were usable, and which parts needed to be rewritten, and provided the client with a proposal for both a quick fix to meet his delivery timelines, as well as a longer term roadmap.
- Based on the findings and action plan, the client decided to terminate the vendor , and hand-over the project to BitLab for implementing the remediation plan.
- The vendor team pushed back, and at times withheld information and source code. However using our reverse engineering practices, we clearly identified the information and source code gaps that were missing, and through negotiations with vendor managed to free-up the necessary artifacts for the team to execute.
Project Results
Within two weeks of taking over we shared the good news with our client. Our first release is 3 weeks away, and a first iteration of the system will be shippable to his customers within a month after that. This would be the quick fixes release.
- November 2020 - BitLab got involved in the project.
- Jan 2021 - First batch of devices ship and deploy on customer sites.
Our client wanted to delight his customers, and spared no expense to make sure they get the most robust system they can get. So our client asked us to also start to work in parallel on the next iteration of the hardware and software with additional functionality.
Don’t just take our word for it
