DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • Observability Architecture: Financial Payments Introduction
  • What ChatGPT Needs Is Context
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Effective Java Collection Framework: Best Practices and Tips

Trending

  • Observability Architecture: Financial Payments Introduction
  • What ChatGPT Needs Is Context
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Effective Java Collection Framework: Best Practices and Tips
  1. DZone
  2. Data Engineering
  3. Databases
  4. Xamarin.Forms — Computer Vision API Using Cognitive Service

Xamarin.Forms — Computer Vision API Using Cognitive Service

This tutorial looks at how to use AI in order to solve business problems.

Delpin Susai Raj user avatar by
Delpin Susai Raj
·
Jul. 09, 18 · Tutorial
Like (2)
Save
Tweet
Share
10.30K Views

Join the DZone community and get the full member experience.

Join For Free

Introduction

Image title

Xamarin.Forms code runs on multiple platforms, each of which has its own filesystem. This means that reading and writing files are the most easily done tasks using native file APIs on each platform. Alternatively, embedded resources are also a simpler solution to distribute the data files with an app.

Image title

Cognitive Services

Infuse your apps, websites, and bots with intelligent algorithms to see, hear, speak, understand, and interpret your user needs through natural methods of communication. Transform your business with AI today.

Use AI to Solve Business Problems

  1. Vision

  2. Speech

  3. Knowledge

  4. Search

  5. Language

Because the Cognitive Services APIs harness the power of machine learning, we were able to bring advanced intelligence into our product without the need to have a team of data scientists on hand.

For more information.

Prerequisites

  • Visual Studio 2017 (Windows or Mac)

Setting Up a Xamarin.Forms Project

Start by creating a new Xamarin.Forms project. You’ll learn more by going through the steps yourself.

Choose the  Cross-platform App project under Visual C#-->Cross-platform in the New Project dialog.

Image title

Now Select the Blank App and Choose Portable Class Library(PCL).

Subsequently, go to the solution. In there, you get all the files and sources of your project (PCL). Now, select XAML page and double-click to open the MainPage.Xaml page.

You now have a basic Xamarin.Forms app. Click the Play button to try it out.

Get Computer Vision API Key

In this step, get computer vision API key.

Go to this link.

Click try Cognitive Services for free.

Image title

Now, you can choose Computer Vision under Vision APIs. Afterward, click "Get API Key."

Image title

Read the terms, and select your country/region. Afterward, click "Next."

Image title

Now, Login using your preferred account.

Image title

Now that the API key is activated, you can now use it. 

Image title

The trial key is only available for 7 days. If you want a permanent key, refer to this article.

Setting Up the User Interface

Go to MainPage.Xaml and write the following code.

MainPage.xaml

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:local="clr-namespace:XamarinComputerVision"
             x:Class="XamarinComputerVision.MainPage">
    <ContentPage.Content>
        <ScrollView>
        <StackLayout HorizontalOptions="CenterAndExpand" VerticalOptions="StartAndExpand">
            <Image x:Name="imgBanner"></Image>
            <Image x:Name="imgChoosed" HeightRequest="200"></Image>
            <Button x:Name="btnPick" Text="Pick" Clicked="btnPick_Clicked"></Button>
            <Button x:Name="btnTake" Text="Take" Clicked="btnTake_Clicked"></Button>
            <Label Text="Result" x:Name="lblResult"></Label>
        </StackLayout>
        </ScrollView>
    </ContentPage.Content>
</ContentPage>

NuGet Packages

Now, add the following NuGet Packages.

  1. Xam.Plugin.Media

  2. Microsoft.ProjectOxford.Vision

Go to Solution Explorer and select your solution. Right-click and select "Manage NuGet Packages for Solution."

Xam.Plugin.Media

Image title

Microsoft.ProjectOxford.Vision

Image title

Permissions — For Android

In this step give requried permissions to your app. the following permissions must for this app.

  1. CAMERA

  2. READ_EXTERNAL_STORAGE

  3. WRITE_EXTERNAL_STORAGE

AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="1" android:versionName="1.0" package="com.companyname.XamarinComputerVision" android:installLocation="auto">
<uses-sdk android:minSdkVersion="15" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<application android:label="XamarinComputerVision.Android">

<provider android:name="android.support.v4.content.FileProvider" 
android:authorities="com.companyname.XamarinComputerVision.fileprovider" 
android:exported="false" 
android:grantUriPermissions="true">
<meta-data android:name="android.support.FILE_PROVIDER_PATHS" 
android:resource="@xml/file_paths"></meta-data>
</provider>
</application>
</manifest>

In this step, create an XML file under Resource-->xml folder for get file paths.

Go to Solution—>Android —>Right click—>New—>Xml—> file_paths.xml

Now, write the following code to get file paths.

file_paths.xml

<?xml version="1.0" encoding="utf-8" ?>
<paths xmlns:android="http://schemas.android.com/apk/res/android">
  <external-files-path name="my_images" path="Pictures" />
  <external-files-path name="my_movies" path="Movies" />
</paths>

Analyze Image

Now, write the following code for analyzing image using Cognitive Service Vision API.

public async Task<AnalysisResult> GetImageDescription(Stream imageStream)
        {
            VisionServiceClient visionClient = new VisionServiceClient("a338648c0df347c6b3b9e46ea2022fcd", "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0");
            VisualFeature[] features = { VisualFeature.Tags, VisualFeature.Categories, VisualFeature.Description };
            return await visionClient.AnalyzeImageAsync(imageStream, features.ToList(), null);
        }

Pick Image

Now, write the following code to pick an image from your device.

MainPage.xaml.cs

 private async void btnPick_Clicked(object sender, EventArgs e)
        {
            await CrossMedia.Current.Initialize();
            try
            {
                var file = await Plugin.Media.CrossMedia.Current.PickPhotoAsync(new Plugin.Media.Abstractions.PickMediaOptions
                {
                    PhotoSize = Plugin.Media.Abstractions.PhotoSize.Medium
                });
                if (file == null)
                    return;
                imgChoosed.Source = ImageSource.FromStream(() =>
                {
                    var stream = file.GetStream();
                    return stream;
                });
                var result = await GetImageDescription(file.GetStream());
                lblResult.Text = null;
                file.Dispose();
                foreach (string tag in result.Description.Tags)
                {
                    lblResult.Text = lblResult.Text + "\n" + tag;
                }

            }
            catch
            (Exception ex)
            {
                string test = ex.Message;
            }
        }

Click the Play Button to Try It Out

Image title

Take Image

Now, write the following code to Take the image using camera.

MainPage.xaml.cs

private async void btnTake_Clicked(object sender, EventArgs e)
        {
            await CrossMedia.Current.Initialize();
            try
            {
                if (!CrossMedia.Current.IsCameraAvailable || !CrossMedia.Current.IsTakePhotoSupported)
                {
                    await DisplayAlert("No Camera", ":( No camera available.", "OK");
                    return;
                }
                var file = await CrossMedia.Current.TakePhotoAsync(new Plugin.Media.Abstractions.StoreCameraMediaOptions
                {
                    Directory = "Sample",
                    Name = "xamarin.jpg"
                });
                if (file == null)
                    return;
                imgChoosed.Source = ImageSource.FromStream(() =>
                {
                    var stream = file.GetStream();
                    return stream;
                });
                var result = await GetImageDescription(file.GetStream());
                file.Dispose();
                lblResult.Text = null;
                //lblResult.Text = result.Description.Captions.First().Text;
                foreach (string tag in result.Description.Tags)
                {
                    lblResult.Text = lblResult.Text + "\n" + tag;
                }
            }
            catch(Exception ex)
            {
                string test = ex.Message;
            }
        }

Click the Play Button to Try It Out

Image title

Full Code — MainPage.Xaml.cs

MainPage.Xaml.cs

using Microsoft.ProjectOxford.Vision;
using Microsoft.ProjectOxford.Vision.Contract;
using Plugin.Media;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace XamarinComputerVision
{
    public partial class MainPage : ContentPage
    {
        public MainPage()
        {
            InitializeComponent();
            imgBanner.Source = ImageSource.FromResource("XamarinComputerVision.images.banner.png");
            imgChoosed.Source = ImageSource.FromResource("XamarinComputerVision.images.thumbnail.jpg");

        }

        private async void btnPick_Clicked(object sender, EventArgs e)
        {
            await CrossMedia.Current.Initialize();
            try
            {
                var file = await Plugin.Media.CrossMedia.Current.PickPhotoAsync(new Plugin.Media.Abstractions.PickMediaOptions
                {
                    PhotoSize = Plugin.Media.Abstractions.PhotoSize.Medium
                });
                if (file == null)
                    return;
                imgChoosed.Source = ImageSource.FromStream(() =>
                {
                    var stream = file.GetStream();
                    return stream;
                });
                var result = await GetImageDescription(file.GetStream());
                lblResult.Text = null;
                file.Dispose();
                foreach (string tag in result.Description.Tags)
                {
                    lblResult.Text = lblResult.Text + "\n" + tag;
                }

            }
            catch
            (Exception ex)
            {
                string test = ex.Message;
            }
        }

        public async Task<AnalysisResult> GetImageDescription(Stream imageStream)
        {
            VisionServiceClient visionClient = new VisionServiceClient("a338648c0df347c6b3b9e46ea2022fcd", "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0");
            VisualFeature[] features = { VisualFeature.Tags, VisualFeature.Categories, VisualFeature.Description };
            return await visionClient.AnalyzeImageAsync(imageStream, features.ToList(), null);
        }

        private async void btnTake_Clicked(object sender, EventArgs e)
        {
            await CrossMedia.Current.Initialize();
            try
            {
                if (!CrossMedia.Current.IsCameraAvailable || !CrossMedia.Current.IsTakePhotoSupported)
                {
                    await DisplayAlert("No Camera", ":( No camera available.", "OK");
                    return;
                }
                var file = await CrossMedia.Current.TakePhotoAsync(new Plugin.Media.Abstractions.StoreCameraMediaOptions
                {
                    Directory = "Sample",
                    Name = "xamarin.jpg"
                });
                if (file == null)
                    return;
                imgChoosed.Source = ImageSource.FromStream(() =>
                {
                    var stream = file.GetStream();
                    return stream;
                });
                var result = await GetImageDescription(file.GetStream());
                file.Dispose();
                lblResult.Text = null;
                //lblResult.Text = result.Description.Captions.First().Text;
                foreach (string tag in result.Description.Tags)
                {
                    lblResult.Text = lblResult.Text + "\n" + tag;
                }
            }
            catch(Exception ex)
            {
                string test = ex.Message;
            }
        }
    }
}

Click the Play Button to Try It Out.

Image title

I hope you have understood how to use Computer Vision API using Cognitive Service in Xamarin.Forms.

Thanks for reading. Please share comments and feedback.

API xamarin.forms Computer app

Opinions expressed by DZone contributors are their own.

Trending

  • Observability Architecture: Financial Payments Introduction
  • What ChatGPT Needs Is Context
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Effective Java Collection Framework: Best Practices and Tips

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: