Implementing Product Visual Search
A picture is worth a thousand words. See how to turn pictures into product information for users.
Join the DZone community and get the full member experience.
Join For FreeUse Case
We will capture a product image from our developed shopping application using a device camera.
- We will show the returned products list.
Prerequisites
Java JDK 1.8 or higher is recommended.
Android Studio is recommended.
Huawei Android device with HMS core 4.0.0.300 or higher.
Before developing an app, you will need to register as a HUAWEI developer.
Integrating the App Gallery Connect (AGC) SDK.
Implementation
1. Enable the ML kit in Manage APIs.
2. Integrate the following dependencies in the app-level build.gradle.
Java
x
1
// Import the product visual search SDK.
2
implementation 'com.huawei.hms:ml-computer-vision-cloud:2.0.1.300'
3. Add the AGC plugin at the top of the app.gradle file.
Java
xxxxxxxxxx
1
1
apply plugin: 'com.huawei.agconnect'
4. Add the following permissions in the manifest.
Camera permission: android.permission.CAMERA: Obtains real-time images or videos from a camera.
Internet access permission: android.permission.INTERNET: Accesses cloud services on the Internet.
Storage write permission: android.permission.WRITE_EXTERNAL_STORAGE: Upgrades the algorithm version.
Storage read permission: android.permission.READ_EXTERNAL_STORAGE: Reads photos stored on a device.
5. To request camera permission in realtime.
Java
xxxxxxxxxx
1
1
private void requestCameraPermission() {
2
final String[] permissions = new String[] {Manifest.permission.CAMERA};
3
4
if (!ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.CAMERA)) {
5
ActivityCompat.requestPermissions(this, permissions, this.CAMERA_PERMISSION_CODE);
6
return;
7
}
8
}
6. Add the following code in the Application class
Java
xxxxxxxxxx
1
1
public class MyApplication extends Application {
2
3
4
public void onCreate() {
5
super.onCreate();
6
MLApplication.getInstance().setApiKey("API KEY");
7
}
8
}
The API key can be obtained either from AGC or the integrated agcconnect-services.json.
7. To create an analyzer for our product visual search.
Java
xxxxxxxxxx
1
12
1
private void initializeProductVisionSearch() {
2
MLRemoteProductVisionSearchAnalyzerSetting settings = new MLRemoteProductVisionSearchAnalyzerSetting.Factory()
3
// Set the maximum number of products that can be returned.
4
.setLargestNumOfReturns(16)
5
// Set the product set ID. (Contact mlkit@huawei.com to obtain the configuration guide.)
6
// .setProductSetId(productSetId)
7
// Set the region.
8
.setRegion(MLRemoteProductVisionSearchAnalyzerSetting.REGION_DR_CHINA)
9
.create();
10
analyzer
11
= MLAnalyzerFactory.getInstance().getRemoteProductVisionSearchAnalyzer(settings);
12
}
8. To capture an image from a camera.
Java
xxxxxxxxxx
1
1
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
2
3
startActivityForResult(intent, REQ_CAMERA_CODE);
9. Once the image has been captured, the onActivityResult() method will be executed.
Java
xxxxxxxxxx
1
70
1
2
public void onActivityResult(int requestCode, int resultCode, Intent data) {
3
super.onActivityResult(requestCode, resultCode, data);
4
Log.d(TAG, "onActivityResult");
5
if(requestCode == 101) {
6
if (resultCode == RESULT_OK) {
7
Bitmap bitmap = (Bitmap) data.getExtras().get("data");
8
if (bitmap != null) {
9
// Create an MLFrame object using the bitmap, which is the image data in bitmap format.
10
MLFrame mlFrame = new MLFrame.Creator().setBitmap(bitmap).create();
11
mlImageDetection(mlFrame);
12
}
13
14
}
15
16
}
17
}
18
19
private void mlImageDetection(MLFrame mlFrame) {
20
Task> task = analyzer.asyncAnalyseFrame(mlFrame);
21
task.addOnSuccessListener(new OnSuccessListener>() {
22
public void onSuccess(List products) {
23
// Processing logic for detection success.
24
25
displaySuccess(products);
26
27
}})
28
.addOnFailureListener(new OnFailureListener() {
29
public void onFailure(Exception e) {
30
// Processing logic for detection failure.
31
// Recognition failure.
32
try {
33
MLException mlException = (MLException)e;
34
// Obtain the result code. You can process the result code and customize respective messages displayed to users.
35
int errorCode = mlException.getErrCode();
36
// Obtain the error information. You can quickly locate the fault based on the result code.
37
String errorMessage = mlException.getMessage();
38
} catch (Exception error) {
39
// Handle the conversion error.
40
}
41
}
42
});
43
}
44
45
46
private void displaySuccess(List productVisionSearchList) {
47
List productImageList = new ArrayList();
48
String prodcutType = "";
49
for (MLProductVisionSearch productVisionSearch : productVisionSearchList) {
50
Log.d(TAG, "type: " + productVisionSearch.getType() );
51
prodcutType = productVisionSearch.getType();
52
for (MLVisionSearchProduct product : productVisionSearch.getProductList()) {
53
productImageList.addAll(product.getImageList());
54
Log.d(TAG, "custom content: " + product.getCustomContent() );
55
}
56
}
57
StringBuffer buffer = new StringBuffer();
58
for (MLVisionSearchProductImage productImage : productImageList) {
59
String str = "ProductID: " + productImage.getProductId() + "
60
ImageID: " + productImage.getImageId() + "
61
Possibility: " + productImage.getPossibility();
62
buffer.append(str);
63
buffer.append("
64
");
65
}
66
Log.d(TAG , "display success: " + buffer.toString());
67
FragmentTransaction transaction = getFragmentManager().beginTransaction();
68
transaction.replace(R.id.main_fragment_container, new SearchResultFragment(productImageList, prodcutType ));
69
transaction.commit();
70
}
The onSuccess() callback will give us a list of MLProductVisionSearch objects, which can be used to get the product id and image URL of each product. Also, we can get the product type using productVisionSearch.getType(). getType(), which returns numbers that can be mapped.
10. We can achieve the product type mapping with the following code.
Java
xxxxxxxxxx
1
25
1
private String getProductType(String type) {
2
switch(type) {
3
case "0":
4
return "Others";
5
case "1":
6
return "Clothing";
7
case "2":
8
return "Shoes";
9
case "3":
10
return "Bags";
11
case "4":
12
return "Digital & Home appliances";
13
case "5":
14
return "Household Products";
15
case "6":
16
return "Toys";
17
case "7":
18
return "Cosmetics";
19
case "8":
20
return "Accessories";
21
case "9":
22
return "Food";
23
}
24
return "Others";
25
}
11. To get product id and image URL from MLVisionSearchProductImage.
Java
xxxxxxxxxx
1
11
1
2
public void onBindViewHolder(ViewHolder holder, int position) {
3
final MLVisionSearchProductImage mlProductVisionSearch = productVisionSearchList.get(position);
4
5
holder.tvTitle.setText(mlProductVisionSearch.getProductId());
6
Glide.with(context)
7
.load(mlProductVisionSearch.getImageId())
8
.diskCacheStrategy(DiskCacheStrategy.ALL)
9
.into(holder.imageView);
10
11
}
Images
Topics:
ai,
machine learning,
java,
image search,
tutorial
Opinions expressed by DZone contributors are their own.
Comments