Skip to main content

Using a Custom Model

MLKit includes a general-purpose image labeling model that recognizes more than 400 entities covering common concepts in photos. However, you may want to use your own custom model for specific use cases.

Compatible Models​

Your custom model needs to be compatible with MLKit's Image Labeling API:

  • The model must be a TensorFlow Lite model (.tflite)
  • The model must be configured for classification tasks
  • The output should provide class labels and confidence scores

Refer to Custom Models with MLKit for detailed information on MLKit model compatibility.

Implementation Steps​

1. Add your model to the project​

Place your model in your project's assets directory:

cp ~/my-custom-model.tflite ./assets/models/my-custom-model.tflite

2. Configure Metro bundler​

Update your metro.config.js to handle .tflite files:

// metro.config.js
const { getDefaultConfig } = require("expo/metro-config");

const config = getDefaultConfig(__dirname);

config.resolver.assetExts.push("tflite");

module.exports = config;

3. Configure the model in your app​

import { ImageLabelingConfig } from '@infinitered/react-native-mlkit-image-labeling';

const MODELS: ImageLabelingConfig = {
  customModel: {
    model: require("./assets/models/my-custom-model.tflite"),
    options: {
      maxResultCount: 5,
      confidenceThreshold: 0.7,
    },
  },
};

4. Use the model in your components​

function CustomModelClassifier() {
  const classifier = useImageLabeling("customModel");

  const classify = async (imageUri: string) => {
    try {
      const results = await classifier.classifyImage(imageUri);
      // Handle classification results
    } catch (error) {
      // Handle errors
    }
  };
}

Model Options​

You can configure your model's behavior using these options:

type CustomImageLabelerOptions = {
  maxResultCount?: number; // Maximum number of classifications to return
  confidenceThreshold?: number; // Minimum confidence score (0-1) for results
};

Updating Model Options​

You can update the model's options at runtime:

await classifier.updateOptionsAndReload({
  maxResultCount: 3,
  confidenceThreshold: 0.8,
});

See the Expo Docs for detailed instructions on customizing metro.