2 Must-Know Tips for Applying TensorFlow Machine Learning

Know these, and applying TensorFlow will be easy peasy

In December 2023, Alarmy (Android) released a machine learning-based snore detection feature. This marked the fifth time we’ve applied TensorFlow Lite machine learning technology to Alarmy.

We went through a lot of trial and error while applying various machine learning technologies, and there were moments when we thought, "If only we had known this earlier..." Today, I'm going to share 2 must-know tips that would have saved us a lot of trouble.

• If a runtime error occurs in the Interpreter, check your TensorFlow Lite library version first.

• Don't use the tensorflow-lite-select-tf-ops library.

1. If a runtime error occurs in the Interpreter, check the tensorflow-lite library version first

You will never solve the error if you don't check version compatibility.

The Interpreter is a component used in TensorFlow Lite to execute and run inferences on TensorFlow Lite models.

When we first applied the sleep analysis machine learning model to Alarmy (Android), a runtime error occurred in the Interpreter.

The error message said we were sending an invalid argument to NNAPI, but we had no idea why. We tried changing the Interpreter settings and tweaking the Input and Output formats, but the error persisted. (To be honest, there aren't that many things you can try on Android.)

It wasn't until we received the Android Project that our modeler was using for testing that we finally figured out the cause. The root of the error was a compatibility issue between the TensorFlow Lite library version and the machine learning model. The model was using an operator that wasn't supported by the tensorflow-lite v2.3.0 version Alarmy was using, and the error triggered the moment the model tried to access that operator. It was only after upgrading to tensorflow-lite v2.13.0 that the errors stopped.

Fortunately, we were able to resolve this incident together with our modeler, but if you're in a situation where you need to track down the cause on your own, you can check compatibility through the following sites:

• NETRON: A site that visually maps out the operators your model is using.

• TensorFlow Lite and TensorFlow operator compatibility: A site where you can check the list of TensorFlow Lite operators.

• TensorFlow Release Note: TensorFlow's GitHub release notes.

If you run into an Interpreter runtime error while applying a TensorFlow model for the first time, don't ask questions—just check your TensorFlow Lite version first.

2. Don't use the tensorflow-lite-select-tf-ops library

It will inflate your app size by up to 110MB.

The tensorflow-lite-select-tf-ops library helps execute complex operations not provided by TensorFlow Lite and improves model compatibility.
However, using it can lead to increased app size and degraded performance.

The initial sleep analysis model used operators not supported by TensorFlow Lite. Since these operators could only be used via the tensorflow-lite-select-tf-ops library, we implemented the ops library without giving it much thought.

The machine learning model functioned perfectly afterward, but a problem popped up in an unexpected place. Alarmy's AAB file size had exceeded the maximum limit allowed for Google Play Console uploads (150MB). The culprit behind the file size inflation was none other than the tensorflow-lite-select-tf-ops library. After applying this library, the AAB file size had jumped by roughly 110MB.

Considering we needed to release the app, removing the tensorflow-lite-select-tf-ops library was inevitable. Dropping the library itself was easy, but the real question was: how do we replace the operators that relied on it?

Thankfully, there was already code on the internet manually implementing these unsupported tensorflow-lite operators, and we confirmed that using it wouldn't cause any major performance issues. By manually implementing the operations directly within the model, we removed the dependency between the tensorflow-lite-select-tf-ops library and our model. We restored our AAB file to its normal size and safely deployed Alarmy.

Aside from manually implementing operators directly in the model, here are other ways to solve the ops library size issue:

• Extracting only the necessary operators from the ops library to create a custom ops library.

• Narrowing down the CPU architecture scope using abiFilters.

The best-case scenario is for the model to exclusively use operators provided by tensorflow-lite natively. Therefore, I highly recommend sharing this tensorflow-lite-select-tf-ops library size issue during the model design phase and proposing that the model be developed without relying on the ops library.

Wrapping up

Trying something new always comes with its fair share of struggles. But since hard work eventually pays off, it's always best to minimize the struggling part as much as possible. I hope the tips I shared today can help lessen your struggles even a little bit.