Classification of sounds using android mobile phone and the YAMNet ML model

  1. Mobile’s microphone records the sound that is converted into an array of floats.
  2. The array is passed to the interpreter that uses the model file which is stored inside assets folder.
  3. The interpreter generates 3 outputs. Scores, embeddings and spectrograms.
  4. The output scores are used to get the top-K classes of the inference.
  5. The top classes are displayed on screen.
val floatsForInference = FloatArray(arrayListShorts.size)
for ((index, value) in arrayListShorts.withIndex()) {
floatsForInference[index] = (value / 32768F)
}
val arrayMeanScores = FloatArray(521) { 0f }
for (i in 0 until 521) {
// Find the average of the 4 arrays at axis = 0
arrayMeanScores[i] = arrayListOf(
arrayScores[0][i],
arrayScores[1][i],
arrayScores[2][i],
arrayScores[3][i]
).average().toFloat()
}
implementation('org.tensorflow:tensorflow-lite-support:0.0.0-nightly')
val labels = FileUtil.loadLabels(context, "classes.txt")
Speech
Child speech, kid speaking
Conversation
Narration, monologue
Babbling
Speech synthesizer
Shout
Bellow
Whoop
Yell
Children shouting
Screaming
Whispering
Laughter
Baby laughter
Giggle
Snicker
Belly laugh
………………………
  1. TensorFlow Support Library usage.
  2. TensorFlow Lite interpreter usage and also:
  3. Databinding
  4. MVVM with Coroutines
  5. Koin DI
  • Tune audio recording in less than 2 seconds and find out the optimal value in time that gives faster results with great accuracy .
  • Add metadata to tflite file so it can be used with ML Binding

--

--

--

I am a pharmacist turned android developer and machine learning engineer. Right now I’m a senior android developer at Invisalign and a ML GDE.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

SAP Vs Oracle: Which ERP System is directly for you?:

Chingu Weekly Vol. 59 — Featuring glimpses from remote dev team collaboration

Join ScooNews hosted Free Webinar addressed by the champions of their fields. REGISTER NOW!

How to manage CROD tokens?

The Home Stretch

A case of migrating towards SignalFX streaming monitoring

The Andon cord and ITSM’s DevOps challenge

Introduction To KiCAD

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
George Soloupis

George Soloupis

I am a pharmacist turned android developer and machine learning engineer. Right now I’m a senior android developer at Invisalign and a ML GDE.

More from Medium

Solution to Creating an Image Classifier

PeduliLindungi: An Alternative Approach

Intermediate: Schedule Alarm in HMS Based Android App

Top 7 Books For Android App Development