Using a Custom Model
MLKit includes a general-purpose image labeling model that recognizes more than 400 entities covering common concepts in photos. However, you may want to use your own custom model for specific use cases.
Compatible Models
Your custom model needs to be compatible with MLKit's Image Labeling API:
- The model must be a TensorFlow Lite model (.tflite)
- The model must be configured for classification tasks
- The output should provide class labels and confidence scores
Refer to Custom Models with MLKit for detailed information on MLKit model compatibility.
Implementation Steps
1. Add your model to the project
Place your model in your project's assets directory:
cp ~/my-custom-model.tflite ./assets/models/my-custom-model.tflite
2. Configure Metro bundler
Update your metro.config.js to handle .tflite files:
// metro.config.js
const { getDefaultConfig } = require("expo/metro-config");
const config = getDefaultConfig(__dirname);
config.resolver.assetExts.push("tflite");
module.exports = config;
3. Configure the model in your app
import { ImageLabelingConfig } from '@infinitered/react-native-mlkit-image-labeling';
const MODELS: ImageLabelingConfig = {
customModel: {
model: require("./assets/models/my-custom-model.tflite"),
options: {
maxResultCount: 5,
confidenceThreshold: 0.7,
},
},
};
4. Use the model in your components
function CustomModelClassifier() {
const classifier = useImageLabeling("customModel");
const classify = async (imageUri: string) => {
try {
const results = await classifier.classifyImage(imageUri);
// Handle classification results
} catch (error) {
// Handle errors
}
};
}
Model Options
You can configure your model's behavior using these options:
type CustomImageLabelerOptions = {
maxResultCount?: number; // Maximum number of classifications to return
confidenceThreshold?: number; // Minimum confidence score (0-1) for results
};
Updating Model Options
You can update the model's options at runtime:
await classifier.updateOptionsAndReload({
maxResultCount: 3,
confidenceThreshold: 0.8,
});
See the Expo Docs for detailed instructions on customizing metro.