Ultra-Simple Integration With ML Kit to Implement Word Broadcasting
Learn how to use the general text recognition and speech synthesis functions of ML Kit to implement an automatic voice broadcast app
Join the DZone community and get the full member experience.
Join For FreeI believe that we all start to learn a language when we start to speak it. When primary school students learn a language, an important part of their homework work is dictating the text of the new words. Many parents know this experience.
On one hand, the pronunciation is relatively simple. On the other hand, parents' time is very precious. Now, there are many dictation voices in the market. These broadcasters record the dictation words in the language teaching materials after class for parents to download. However, this kind of recording is not flexible enough. If the teacher leaves a few extra words that are not part of the after-school problem set, the recordings won't meet the needs of parents and children.
This article describes how to use the general text recognition and speech synthesis functions of ML Kit to implement an automatic voice broadcast app. You only need to take photos of dictation words or texts, and then the text in the photos can be automatically played. The tone of the voice can be adjusted.
Development Preparations
Open the project-level build.gradle file:
Choose allprojects > repositories and configure the Maven repository address of HMS SDK:
Java
x
1
allprojects {
2
repositories {
3
google()
4
jcenter()
5
maven {url 'http://developer.huawei.com/repo/'}
6
}
7
}
Configure the Maven repository address of HMS SDK in buildscript->repositories:
Java
xxxxxxxxxx
1
1
buildscript {
2
repositories {
3
google()
4
jcenter()
5
maven {url 'http://developer.huawei.com/repo/'}
6
}
7
}
Choose buildscript > dependencies and configure the AGC plug-in:
Java
xxxxxxxxxx
1
1
dependencies {
2
classpath 'com.huawei.agconnect:agcp:1.2.1.301'
3
}
Adding Compilation Dependencies
Open the application levelbuild.gradle file:
SDK integration:
Java
xxxxxxxxxx
1
1
dependencies{
2
implementation 'com.huawei.hms:ml-computer-voice-tts:1.0.4.300'
3
implementation 'com.huawei.hms:ml-computer-vision-ocr:1.0.4.300'
4
implementation 'com.huawei.hms:ml-computer-vision-ocr-cn-model:1.0.4.300'
5
}
Add the ACG plug-in to the file header:
Java
xxxxxxxxxx
1
1
apply plugin: 'com.huawei.agconnect'
Specify permissions and features: Declare them in AndroidManifest.xml.
Java
xxxxxxxxxx
1
1
<uses-permission android:name="android.permission.CAMERA" />
2
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
3
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
4
<uses-feature android:name="android.hardware.camera" />
5
<uses-feature android:name="android.hardware.camera.autofocus" />
Key Development Steps
There are two main functions. One is to identify the operation text, and the other is to read the operation. The OCR+TTS mode is used to read the operation. After taking a photo, click the play button to read the operation.
1. Dynamic permission application:
Java
xxxxxxxxxx
1
1
private static final int PERMISSION_REQUESTS = 1;
2
3
public void onCreate(Bundle savedInstanceState) {
4
// Checking camera permission
5
if (!allPermissionsGranted()) {
6
getRuntimePermissions();
7
}
8
}
2. Start the reading interface:
Java
xxxxxxxxxx
1
1
public void takePhoto(View view) {
2
Intent intent = new Intent(MainActivity.this, ReadPhotoActivity.class);
3
startActivity(intent);
4
}
3. Invoke createLocalTextAnalyzer() in the onCreate() method to create a device-side text recognizer:
Java
xxxxxxxxxx
1
1
private void createLocalTextAnalyzer() {
2
MLLocalTextSetting setting = new MLLocalTextSetting.Factory()
3
.setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
4
.setLanguage("zh")
5
.create();
6
this.textAnalyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting);
7
8
}
4. Invoke createLocalTextAnalyzer() in the onCreate() method to create a device-side text recognizer:
Java
xxxxxxxxxx
1
32
1
private void createTtsEngine() {
2
MLTtsConfig mlConfigs = new MLTtsConfig()
3
.setLanguage(MLTtsConstants.TTS_ZH_HANS)
4
.setPerson(MLTtsConstants.TTS_SPEAKER_FEMALE_ZH)
5
.setSpeed(0.2f)
6
.setVolume(1.0f);
7
this.mlTtsEngine = new MLTtsEngine(mlConfigs);
8
MLTtsCallback callback = new MLTtsCallback() {
9
10
public void onError(String taskId, MLTtsError err) {
11
}
12
13
14
public void onWarn(String taskId, MLTtsWarn warn) {
15
}
16
17
18
public void onRangeStart(String taskId, int start, int end) {
19
}
20
21
22
public void onEvent(String taskId, int eventName, Bundle bundle) {
23
if (eventName == MLTtsConstants.EVENT_PLAY_STOP) {
24
if (!bundle.getBoolean(MLTtsConstants.EVENT_PLAY_STOP_INTERRUPTED)) {
25
Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.read_finish, Toast.LENGTH_SHORT).show();
26
}
27
}
28
}
29
};
30
mlTtsEngine.setTtsCallback(callback);
31
}
5. Set the buttons for reading photos, taking photos, and reading aloud.
Java
xxxxxxxxxx
1
13
1
this.relativeLayoutLoadPhoto.setOnClickListener(new View.OnClickListener() {
2
3
public void onClick(View v) {
4
ReadPhotoActivity.this.selectLocalImage(ReadPhotoActivity.this.REQUEST_CHOOSE_ORIGINPIC);
5
}
6
});
7
this.relativeLayoutTakePhoto.setOnClickListener(new View.OnClickListener() {
8
9
public void onClick(View v) {
10
ReadPhotoActivity.this.takePhoto(ReadPhotoActivity.this.REQUEST_TAKE_PHOTO);
11
}
12
});
6. Start TextAnalyzer() during the callback of photographing and reading photos:
Java
xxxxxxxxxx
1
28
1
private void startTextAnalyzer() {
2
if (this.isChosen(this.originBitmap)) {
3
MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();
4
Task<MLText> task = this.textAnalyzer.asyncAnalyseFrame(mlFrame);
5
task.addOnSuccessListener(new OnSuccessListener<MLText>() {
6
7
public void onSuccess(MLText mlText) {
8
// Transacting logic for segment success.
9
if (mlText != null) {
10
ReadPhotoActivity.this.remoteDetectSuccess(mlText);
11
} else {
12
ReadPhotoActivity.this.displayFailure();
13
}
14
}
15
}).addOnFailureListener(new OnFailureListener() {
16
17
public void onFailure(Exception e) {
18
// Transacting logic for segment failure.
19
ReadPhotoActivity.this.displayFailure();
20
return;
21
}
22
});
23
} else {
24
Toast.makeText(this.getApplicationContext(), R.string.please_select_picture, Toast.LENGTH_SHORT).show();
25
return;
26
}
27
}
7. After the recognition is successful, click the play button to start the playback:
Java
xxxxxxxxxx
1
12
1
this.relativeLayoutRead.setOnClickListener(new View.OnClickListener() {
2
3
public void onClick(View v) {
4
if (ReadPhotoActivity.this.sourceText == null) {
5
Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.please_select_picture, Toast.LENGTH_SHORT).show();
6
} else {
7
ReadPhotoActivity.this.mlTtsEngine.speak(sourceText, MLTtsEngine.QUEUE_APPEND);
8
Toast.makeText(ReadPhotoActivity.this.getApplicationContext(), R.string.read_start, Toast.LENGTH_SHORT).show();
9
}
10
}
11
});
Demo
Topics:
ai,
ml kit,
voice recognization,
machine learning,
tutorial
Opinions expressed by DZone contributors are their own.
Comments