Intermediate: Simplified integration of HMS ML Kit with Object Detection and Tracking API using Xamarin
HMS ML Kit allows your apps to easily leverage Huawei’s long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries.
Overview
In this article, I will create a demo app along with the integration of HMS ML Kit which based on Cross platform Technology Xamarin. User can easily scan any objects from this application with camera using Object Detection and Tracking API and choose best price and details of object. The following object categories are supported: household products, fashion goods, food, places, plants, faces, and others.
Service Introduction
HMS ML Kit allows your apps to easily leverage Huawei’s long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries.
A user can take a photo of an Object through camera or gallery. Then the Object Detection and Tracking service searches for the same or similar objects in the pre-established object image library and returns the IDs of those object and related information.
We can capture or choose from gallery any kind object based image to buy or check the price of an object using Machine Learning. It will give the other options, so that you can improve your buying skills.
Prerequisite
1. Xamarin Framework
2. Huawei phone
3. Visual Studio 2019
App Gallery Integration process
1. Sign In and Create or Choose a project on AppGallery Connect portal.
2. Add SHA-256 key.
3. Navigate to Project settings and download the configuration file.
4. Navigate to General Information, and then provide Data Storage location.
5. Navigate to Manage APIs and enable APIs which require by application.
Xamarin ML Kit Setup Process
1. Download Xamarin Plugin all the aar and zip files from below url:
2. Open the XHms-ML-Kit-Library-Project.sln solution in Visual Studio.
3. Navigate to Solution Explore and right-click on jar Add > Exsiting Item and choose aar file which download in Step 1.
4. Right click on added aar file then choose Properties > Build Action > LibraryProjectZip
Note: Repeat Step 3 & 4 for all aar file.
5. Build the Library and make dll files.
Xamarin App Development
1. Open Visual Studio 2019 and Create A New Project.
2. Navigate to Solution Explore > Project > Assets > Add Json file.
3. Navigate to Solution Explore > Project > Add > Add New Folder.
4. Navigate to Folder(created) > Add > Add Existing and add all DLL files.
5. Select all DLL files.
6. Right-click on Properties, choose Build Action > None.
7. Navigate to Solution Explore > Project > Reference > Right Click > Add References, then navigate to Browse and add all DLL files from recently added folder.
8. Added reference, then click OK.
ML Object Detection and Tracking API Integration
Camera stream detection
You can process camera streams, convert video frames into an MLFrame object, and detect objects using the static image detection method. If the synchronous detection API is called, you can also use the LensEngine class built in the SDK to detect objects in camera streams. The sample code is as follows:
- Create an object analyzer.
2. Create the ObjectAnalyzerTransactor class for processing detection results. This class implements the MLAnalyzer.IMLTransactor API and uses the TransactResult method in this API to obtain the detection results and implement specific services.
3. Set the detection result processor to bind the analyzer to the result processor.
analyzer.SetTransactor(new ObjectAnalyseMLTransactor());
4. Create an instance of the LensEngine class provided by the HMS Core ML SDK to capture dynamic camera streams and pass the streams to the analyzer.
5. Call the run method to start the camera and read camera streams for detection.
6. After the detection is complete, stop the analyzer to release detection resources.
LiveObjectAnalyseActivity.cs
This activity performs all the operation regarding object detecting and tracking with camera.
Xamarin App Build
1. Navigate to Solution Explore > Project > Right Click > Archive/View Archive to generate SHA-256 for build release and Click on Distribute.
2. Choose Distribution Channel > Ad Hoc to sign apk.
3. Choose Demo Keystore to release apk.
4. Finally here is the Result.
Tips and Tricks
1. HUAWEI ML Kit complies with GDPR requirements for data processing.
2. HUAWEI ML Kit does not support the recognition of the object distance and colour.
3. Images in PNG, JPG, JPEG, and BMP formats are supported. GIF images are not supported.
Conclusion
In this article, we have learned how to integrate HMS ML Kit in Xamarin based Android application. User can easily search objects online with the help of Object Detection and Tracking API in this application.
Thanks for reading this article.
Be sure to like and comments to this article, if you found it helpful. It means a lot to me.
References
