DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workkloads.

Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Stateless vs. Stateful Widgets: Make the Right Choice for Your Flutter App
  • Build Your Own AI Avatar App From Scratch in Less Than 8 Hours
  • The Technology Stack Needed To Build a Web3 Application
  • How To Use IBM App Connect To Build Flows

Trending

  • Mastering Fluent Bit: Installing and Configuring Fluent Bit on Kubernetes (Part 3)
  • Transforming AI-Driven Data Analytics with DeepSeek: A New Era of Intelligent Insights
  • A Modern Stack for Building Scalable Systems
  • Apache Doris vs Elasticsearch: An In-Depth Comparative Analysis

Build an Emoji-Making App Effortlessly

Build a custom emoji-making app that captures users' facial expressions and makes conversations more interesting.

By 
Jackson Jiang user avatar
Jackson Jiang
DZone Core CORE ·
Sep. 19, 22 · Tutorial
Likes (1)
Comment
Save
Tweet
Share
3.5K Views

Join the DZone community and get the full member experience.

Join For Free

Emojis are a must-have tool in today's online communications as they help add color to text-based chatting and allow users to better express the emotions behind their words. Since the number of preset emojis is always limited, many apps now allow users to create their own custom emojis to keep things fresh and exciting.

For example, in a social media app, users who do not want to show their faces when making video calls can use an animated character to protect their privacy, with their facial expressions applied to the character; in a live streaming or e-commerce app, virtual streamers with realistic facial expressions are much more likely to attract watchers; in a video or photo shooting app, users can control the facial expressions of an animated character when taking a selfie, and then share the selfie via social media; and in an educational app for kids, a cute animated character with detailed facial expressions will make online classes much more fun and engaging for students.

I myself am developing such a messaging app. When chatting with friends and wanting to express themselves in ways other than words, users of my app can take a photo to create an emoji of themselves, or of an animated character they have selected. The app will then identify users' facial expressions, and apply their facial expressions to the emoji. In this way, users are able to create an endless amount of unique emojis. During the development of my app, I used the capabilities provided by HMS Core AR Engine to track users' facial expressions and convert the facial expressions into parameters, which greatly reduced the development workload. Now I will show you how I managed to do this.

Implementation

AR Engine provides apps with the ability to track and recognize facial expressions in real time, which can then be converted into facial expression parameters and used to accurately control the facial expressions of virtual characters.

Currently, the kit provides 64 facial expressions, including eyelid, eyebrow, eyeball, mouth, and tongue movements. It supports 21 eye-related movements, including eyeball movement and opening and closing the eyes; 28 mouth movements, including opening the mouth, puckering, pulling, or licking the lips, and moving the tongue; as well as five eyebrow movements, including raising or lowering the eyebrows.

Demo

Facial expression-based emoji

Development Procedure

Requirements on the Development Environment

JDK: 1.8.211 or later

Android Studio: 3.0 or later

minSdkVersion: 26 or later

targetSdkVersion: 29 (recommended)

compileSdkVersion: 29 (recommended)

Gradle version: 6.1.1 or later (recommended)

Make sure that you have downloaded the AR Engine APK from AppGallery and installed it on the device.

If you need to use multiple HMS Core kits, use the latest versions required for these kits.

Preparations

1. Before getting started, you will need to register as a Huawei developer and complete identity verification. You can click here to find out the detailed registration and identity verification procedure.

2. Before development, integrate the AR Engine SDK via the Maven repository into your development environment.

3. The procedure for configuring the Maven repository address in Android Studio varies for Gradle plugin earlier than 7.0, Gradle plugin 7.0, and Gradle plugin 7.1 or later. You need to configure it according to the specific Gradle plugin version.

4. Take Gradle plugin 7.0 as an example:

Open the project-level build.gradle file in your Android Studio project and configure the Maven repository address.

Go to buildscript > repositories and configure the Maven repository address for the SDK.

 
buildscript {
     repositories {
         google()
         jcenter()
         maven {url "https://developer.huawei.com/repo/" }
     }
}

Open the project-level settings.gradle file and configure the Maven repository address for the HMS Core SDK.

 
dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
      repositories {
           repositories {
                google()
               jcenter()
               maven {url "https://developer.huawei.com/repo/" }
           }
       }
}

5. Add the following build dependency in the dependencies block.

 
dependencies {
    implementation 'com.huawei.hms:arenginesdk:{version}
}

App Development

1. Check whether AR Engine has been installed on the current device. If yes, your app can run properly. If not, you need to prompt the user to install it, for example, by redirecting the user to AppGallery. The sample code is as follows:

 
boolean isInstallArEngineApk =AREnginesApk.isAREngineApkReady(this);
if (!isInstallArEngineApk) {
    // ConnectAppMarketActivity.class is the activity for redirecting users to AppGallery.
startActivity(new Intent(this, com.huawei.arengine.demos.common.ConnectAppMarketActivity.class));
    isRemindInstall = true;
}

2. Create an AR scene. AR Engine supports five scenes, including motion tracking (ARWorldTrackingConfig), face tracking (ARFaceTrackingConfig), hand recognition (ARHandTrackingConfig), human body tracking (ARBodyTrackingConfig), and image recognition(ARImageTrackingConfig).

The following takes creating a face tracking scene by calling ARFaceTrackingConfig as an example.

 
// Create an ARSession object.
mArSession = new ARSession(this);
// Select a specific Config to initialize the ARSession object based on the application scenario.
ARFaceTrackingConfig config = new ARFaceTrackingConfig(mArSession);

Set scene parameters using the config.setXXX method.

 
// Set the camera opening mode, which can be external or internal. The external mode can only be used in ARFace. Therefore, you are advised to use the internal mode.
mArConfig.setImageInputMode(ARConfigBase.ImageInputMode.EXTERNAL_INPUT_ALL);

3. Set the AR scene parameters for face tracking and start face tracking.

 
mArSession.configure(mArConfig);
mArSession.resume();

4. Initialize the FaceGeometryDisplay class to obtain the facial geometric data and render the data on the screen.

 
public class FaceGeometryDisplay {
// Initialize the OpenGL ES rendering related to face geometry, including creating the shader program.
void init(Context context) {...
}
}

5. Initialize the onDrawFrame method in the FaceGeometryDisplay class, and call face.getFaceGeometry() to obtain the face mesh.

 
public void onDrawFrame(ARCamera camera, ARFace face) {
    ARFaceGeometry faceGeometry = face.getFaceGeometry();
    updateFaceGeometryData(faceGeometry);
    updateModelViewProjectionData(camera, face);
    drawFaceGeometry();
    faceGeometry.release();
}

6. Initialize updateFaceGeometryData() in the FaceGeometryDisplay class.

Pass the face mesh data for configuration and set facial expression parameters using OpenGl ES.

 
private void updateFaceGeometryData (ARFaceGeometry faceGeometry) {
FloatBuffer faceVertices = faceGeometry.getVertices();
FloatBuffer textureCoordinates =faceGeometry.getTextureCoordinates();
// Obtain an array consisting of face mesh texture coordinates, which is used together with the vertex data returned by getVertices() during rendering.
}

7. Initialize the FaceRenderManager class to manage facial data rendering.

 
public class FaceRenderManager implements GLSurfaceView.Renderer {
public FaceRenderManager(Context context, Activity activity) {
    mContext = context;
    mActivity = activity;
}
// Set ARSession to obtain the latest data.
public void setArSession(ARSession arSession) {
    if (arSession == null) {
        LogUtil.error(TAG, "Set session error, arSession is null!");
        return;
    }
    mArSession = arSession;
}
// Set ARConfigBase to obtain the configuration mode.
public void setArConfigBase(ARConfigBase arConfig) {
    if (arConfig == null) {
        LogUtil.error(TAG, "setArFaceTrackingConfig error, arConfig is null.");
        return;
    }
    mArConfigBase = arConfig;
}
// Set the camera opening mode.
public void setOpenCameraOutsideFlag(boolean isOpenCameraOutsideFlag) {
    isOpenCameraOutside = isOpenCameraOutsideFlag;
}
...
@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
mFaceGeometryDisplay.init(mContext);
}
}

8. Implement the face tracking effect by calling methods like setArSession and setArConfigBase of FaceRenderManager in FaceActivity.

 
public class FaceActivity extends BaseActivity {
@Override
protected void onCreate(Bundle savedInstanceState) {
mFaceRenderManager = new FaceRenderManager(this, this);
mFaceRenderManager.setDisplayRotationManage(mDisplayRotationManager);
mFaceRenderManager.setTextView(mTextView);

glSurfaceView.setRenderer(mFaceRenderManager);
glSurfaceView.setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY);
}
}

Conclusion

Emojis allow users to express their moods and excitement in a way words can't. Instead of providing users with a selection of the same old boring preset emojis that have been used a million times, you can now make your app more fun by allowing users to create emojis themselves! Users can easily create an emoji with their own smiles, simply by facing the camera, selecting an animated character they love, and smiling. With such an ability to customize emojis, users will be able to express their feelings in a more personalized and interesting manner. 

If you have any interest in developing such an app, AR Engine is one potential option. It is able to identify users' facial expressions in real time, convert the facial expressions into parameters, and then apply them to virtual characters. Integrating the capability can help you considerably streamline your app development process, leaving you with more time to focus on how to provide more interesting features to users and improve your app's user experience.

app Build (game engine)

Published at DZone with permission of Jackson Jiang. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Stateless vs. Stateful Widgets: Make the Right Choice for Your Flutter App
  • Build Your Own AI Avatar App From Scratch in Less Than 8 Hours
  • The Technology Stack Needed To Build a Web3 Application
  • How To Use IBM App Connect To Build Flows

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!