Fair warning — this is an awesomely nerdy blog post. If you just want to learn what Uploader 2.0 is all about and install it, go here.
If you want a behind the scenes look at how and why we built it, this post is for you!
TL;DR
Interview with the Engineers behind Tidepool Uploader 2.0
I asked Gerrit Niezen and Chris McGee some rather basic questions, and asked them to be as detailed as they wanted.
Gerrit & Chris: The Uploader does the heavy lifting getting the data off of compatible diabetes devices and transmitting the data to our servers where it’s securely stored. It requests data from devices using the USB port, and processes the data into a common format known as the Tidepool data model. Almost every device out there has its own unique way of storing and representing diabetes data, so we have to convert the data into a common representation to ensure that apps, like Tidepool, can display the data from various devices in the same way. The Uploader also takes the various timezones and time changes on the device into account - see this blog post for more detail.
The first version of our Uploader was a Chrome App, not to be confused with a Chrome Extension - which you’d see next to the address bar. Chrome Apps have their own User Interface (UI), whereas a Chrome Extension merely augments the browser experience. More importantly, Chrome Apps get more access to the machine that the browser is running on than regular web apps, which was particularly advantageous for us because we were able to leverage Chrome’s native USB support for our software to work.
This USB support comes in the form of APIs for two different interfaces: Serial and Human Interface Device (HID). The diabetes devices that we want to communicate with use either of these two interfaces to communicate to computers. Because Chrome Apps have that support built in, we didn’t have to write the low level communication software to make sure we were compatible with Windows and macOS operating systems, we just had to make sure we worked with Google Chrome, and we were good to go on both Windows and MacOS.
This also created some limitations and restrictions around our software, but we felt the advantages of creating a Chrome App, most notably the built-in USB communication support, was a huge advantage to us developing our first Uploader. Additionally, Chrome Apps are relatively easy to distribute through the Chrome Store. So when we published an update, that new code was propagated out to our users seamlessly and automatically.
There were two issues.
New Tidepool users sometimes struggled to install the app because getting a Chrome App was a new, and different, experience.
But adding fuel to the fire, Google announced they will no longer support Chrome Apps unless they are running on a Chromebook, running ChromeOS. This means we had to find a solution for the Tidepool Uploader that no longer relied on Google Chrome, the Chrome Store, and Chrome Apps.
Google announced this decision last year, you can read more on their blog - https://blog.chromium.org/2016/08/from-chrome-apps-to-web.html. While there was no firm stop date announced, we knew we had to get to work sooner rather than later to get a new solution developed, tested, verified, and ready for the diabetes community.
USB drivers, which are often times proprietary, makes it easier for software to communicate with these diabetes devices over the Serial or HID interfaces instead of over the raw USB interface. That driver is usually installed in Kernel space (technically, this term and location varies depending on operating system), where it has access to talk directly to hardware.
The thing about drivers, though, is that you need Administrative privileges to install them because drivers have access to the hardware on your computer. Drivers operate in a privileged execution space within your Operating System, so there needs to be an additional layer of security around their installation and maintenance.
But once that installation is authorized by something like an Administrator password, the Operating System largely stays out of the way of a new driver. All of this means you have to trust the source of these types of files. Chipset manufacturers, that build the USB microchips used by the device manufacturers in their diabetes devices, digitally sign their USB drivers. We also digitally sign the Tidepool Installer used to install the USB drivers. Unsigned software is untrusted, and can be misused to damage a machine.
We’ve gone to great lengths to ensure that we could have a single, installable driver package that would cover support for all Tidepool-compatible devices like Animas, Dexcom, Omnipod, Medtronic, and others. Being able to redistribute these drivers in a single package makes things much easier for us. Rather than requiring users to go to a handful of different locations or websites to download specific drivers for their different diabetes devices, we bundle all the drivers into a single file - if we support a device, its driver is included in our Driver Package for the Tidepool Uploader.
Connecting your insulin pump to your computer with a USB cable is meaningless to the Tidepool Uploader without the drivers to facilitate data being transferred. Without the Driver Package, the communication to your diabetes devices does not exist and none of this is possible.
The short and sweet answer is, we couldn't put drivers into a Chrome App. That’s a development restriction put in place, in part, by Google. Chrome Apps are installed into User Space, which created specific limitations on how far our access as engineers went. Chrome Apps have access only as far as Google Chrome permits.
By requiring our users to install the separate Driver Package, we were able to communicate with the devices from inside of the Chrome App and you could access your diabetes data.
In order to replicate the user experience of the Chrome App, we pieced together components of the Tidepool Uploader through a framework called Electron. These components - Chromium, Node.js, and V8 - form the foundation of the Tidepool Uploader.
Chromium is the open source project that serves as the foundation for Google Chrome. For all intents and purposes, Chromium is a web browser, but it lacks a few of the additional features that Google Chrome has like account syncing and USB communications. (Remember how USB communications is a specific feature available to Google Chrome and Chrome Apps? That’s not available on Chromium.)
We rely on Chromium for the visual experience of the Uploader. Everything you see and click on is handled by Chromium. But without the USB APIs that were available to Chrome Apps included in Google Chrome, an additional component is necessary to allow the Tidepool Uploader to communicate with all of the devices we support. This brings us to node.js.
Node.js is an operating system-level execution environment for JavaScript. Through Node.js, we are able to do a lot of the things behind the scenes that users do not see that we were previously doing with the Chrome App - most importantly, talk to USB devices.
Finally, Electron includes V8, the JavaScript execution engine that powers all of our code.
All of this means the Uploader can do all of the things it was doing before when it was just a Chrome App, but now we are no longer held back by restrictions placed on Google Chrome. While you still have to use Google Chrome to view your data, the act of uploading your diabetes data happens independent of Google Chrome.
Node.js forms part of the JavaScript execution and allows us to replicate the driver capabilities that we used to have in our previous Uploader through Google Chrome. Now that we are using the Electron framework for the new Tidepool Uploader, Chromium is able to handle the visual components of the Uploader, but we use node.js libraries to communicate with diabetes devices over USB.
The key thing about node.js is that it enables JavaScript to be run outside a web browser, whereas most web browsers already run JavaScript on their own. Additionally, Node.js actually runs on the same engine that Google Chrome and Chromium use. Which, if you sort all this out, means that what we’ve built to work in Google Chrome (which runs on V8, and is built upon Chromium), can be replicated without the Google Chrome experience by relying on Node.js.
Now, we have full control over the installation and setup process for our software. For our users, there is only one file to download and install. The Driver Package, which was previously a separate file, is now bundled with the rest of the Uploader. Part of the Uploader installation process includes running the driver installer. So when we add a new compatible devices to the Tidepool Uploader, users will get a notification for an update and can install the latest version like they would any other software update.
For us, making updates will be much more streamlined. With the Chrome App Store, we had to follow very specific protocols for updating the app. Now, we have more control, leveraging GitHub’s Releases functionality to indicate and propagate updates. The Tidepool Uploader will check for new updates once a day, and every time you launch the app, so there shouldn’t be a risk of missing out on new features or functionality.
Now we are able to create an application that one would think would normally run on a browser, but we have much more intimate hardware communication potential through a stand-alone application, which is what we’ve built with Electron. While things are otherwise unchanged for our users, all of the code we are writing is leveraging more powerful APIs through Node.js than we previously had access to in creating a Chrome App.
There are a lot of little things we have access to now that we’re building a stand-alone app with Electron. And, there were a lot of things we did with our Chrome App that didn’t make a lot of sense from a “normal web application” development standpoint. A lot of our design and engineering decisions were because we were bound by the Google Chrome and Chrome App environment.
For instance, having an actual page URL is something you don’t have with a Chrome App, but we have that opportunity now in an Electron environment. We aren’t surfacing the URL to users, but when it comes to debugging and troubleshooting, leveraging a URL with parameters being passed from screen to screen will be tremendously helpful if we have a bug to replicate and eventually squash.
URLs mean we have proper navigation through the Uploader experience, this means we can be more precise with updates to the User Experience, which means adding a new page or splash screen is a much easier process. While moving to a true page system will help us a lot in the future, it did represent a fair amount of work separating the components to their proper home in the new codebase.
Because Google Chrome and Electron are using the same rendering engine, Chromium, we didn’t have to change too much to ensure the same visual experience for Tidepool users. But a lot of the outer framework of the Uploader did need to be modified quite a bit. Now that we aren’t held back by size restrictions for Chrome Apps, we can play with space much more liberally as development on the Uploader continues. Creating a more responsive application opens up more opportunities to improve and expand what the Uploader, and the broader Tidepool platform can accomplish within a single application.
Feedback from Tidepool users and the broader diabetes community was the driving force behind our improved support for Medtronic insulin pumps. After we launched support for Paradigm 523 and 723, and MiniMed 530G, we heard the same key message, “I’m glad Tidepool works with my insulin pump, but I wish the data upload process was easier.”
As thrilled as we were to debut our initial support for Medtronic devices, user feedback pushed us to do better. And we knew better had to be a priority for us if we wanted to effectively address the needs of the diabetes community members using Medtronic pumps.
Initially, we supported Medtronic’s Paradigm 523 and 723, 530G and Enlite sensor by requiring users upload their data to Carelink. We were then able to download their data from Carelink, upload it to our secure servers, and convert that data into our data format.
Direct Medtronic uploading is the next phase of our growing support for insulin pump compatibility with Tidepool. The Tidepool Uploader now communicates directly with Medtronic’s Paradigm 523, 723, and 530G pumps to upload data directly from the pump to Tidepool.
In order to utilize Medtronic Direct, users just need a Bayer Contour Next Link meter.
If you’re creating a Tidepool account for the first time, be sure to select “Medtronic Direct from Bayer Contour Next Link” as one of your devices when you are configuring the Tidepool Uploader.
If you already have a Tidepool account, when you log into the new Tidepool Uploader, click the arrow next to your name, select “Choose Devices”, uncheck “Medtronic (from CareLink)”, and select “Medtronic Direct from Bayer Contour Next Link”.
The Tidepool Uploader will ask for a 6-digit serial number on the back of your pump. After you’ve entered the serial number, and connected your Bayer Contour Next Link meter to your computer, you can upload your Medtronic data directly to Tidepool.
Bayer Contour Next Link meters wirelessly communicate with compatible Medtronic insulin pumps, which is a fantastic feature on its own. Also important for us is the fact that the Contour Next Link meters are compatible with USB 2.0 and 3.0. Many folks were having trouble with the grey CareLink stick because it is not compatible with USB 3.0 ports found on most modern computers. Most importantly, Bayer publicly published the protocol specification for their Contour Next Link meters on their website.
We’re constantly evaluating feedback from the Tidepool community to prioritize what updates and features we should be focusing on next. With the release of direct Medtronic support in the Tidepool Uploader, we’re getting ready to launch a public beta for Medtronic 6-series insulin pumps. Anyone interested in participating in our Medtronic 6-series beta should fill out this form.
Absolutely! As always, part of our mission as a nonprofit is to share everything we learn and do with the community. All of Tidepool’s source code is available at https://github.com/tidepool-org
---
There are a lot of these tech-based conversations that happen internally at Tidepool that I’d love to share with you all more regularly. If you have specific questions about the “how” and “why” of Tidepool, please let me know and I’ll ask around.
We'll keep you updated with all the need to know news and announcements for Tidepool and Tidepool Loop
Donate today to support Tidepool's mission to improve the lives of People with Diabetes through affordable, accessible and interoperable technologies
Want to get the latest news and updates about Tidepool and Tidepool Loop? We promise to keep it interesting in your inbox.
