Google Glass OCR Tutorial using Eclipse

Glass OCR Tesseract

Glass OCR Tesseract

While scanning QR codes can be achieved by porting the ZXing library app to Google Glass (which has been accomplished by BarcodeEye), I thought it be interesting to also combine OCR and Glass. OCR stands for Optical Character Recognition. The Wiki definition states that it is the mechanical or electronic conversion of scanned or photographed images of typewritten or printed text into machine-encoded/computer-readable text. I normally refer OCR as the process to read printed texts. In this blog post I will share how I’ve used Google Glass to do OCR. I hope it would be interesting to some of you.

Here are the steps involved:

1. Create Google Glass project in Eclipse.

If you are not familiar with that, it is best to directly follow Google’s GDK Quick Start guide. And because I wanted to add a custom voice command to the Ok Glass menu, I added the following line to the manifest file:

<uses-permission android:name=”com.google.android.glass.permission.DEVELOPMENT” />

2. Build Tesseract.

Here we are using Tesseract for OCR. It provides a set of Android APIs to build the Tesseract Optical Character Recognition and Leptonica image processing libraries.

1) Download Tesseract from Github (this is an excellent version forked by Robert Theis call tess-two)

2) Download android-ndk which allows Tesseract to build at http://developer.android.com/tools/sdk/ndk/index.html 

3) Extract it to a folder (e.g. c:\Software\android-ndk)

4) Set it in your Environment Variable. Go to Control Panel -> Environment Variable, add your extracted folder path to the variable PATH. (e.g. c:\Software\android-ndk)

5) Open command prompt, and go into your project from step one

6) Type “ndk-build”, and the project will compile. It takes about an hour.

3. Add Tesseract as a library.

Open the properties of your project created in Step one, add tess-two library as a reference.

Android add project library

4. Add the following sample code to the project’s main activity.

String thumbnailFilePath = extras.getString(CameraManager.EXTRA_THUMBNAIL_FILE_PATH);
String pictureFilePath = extras.getString(CameraManager.EXTRA_PICTURE_FILE_PATH);
BitmapFactory.Options options = new BitmapFactory.Options();
options.inSampleSize = 4;
Bitmap bitmap = BitmapFactory.decodeFile(thumbnailFilePath, options);
bitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
String lang = "eng";
//Make sure this path exist
String DATA_PATH = Environment .getExternalStorageDirectory().toString() + "/Android/data/";
TessBaseAPI baseApi = new TessBaseAPI();
baseApi.setDebug(true); baseApi.init(DATA_PATH, "eng");
baseApi.setImage(bitmap);
String recognizedText = baseApi.getUTF8Text();

That should do it and you can download the sample code at Github for further customizations and extensions.

This is Michael Siu from Idea Notion, a consulting firm that develops enterprise web and mobile apps. If you have any questions, would like to learn more, or are interested in developing Google Glass apps, feel free to contact me. Follow our Google Glass Series to find out what exciting things we are building with Google Glass.

By Michael Siu

Posted in Glass, Technical
17 comments on “Google Glass OCR Tutorial using Eclipse
  1. hello can you help me to configure google teseract project
    When i create object using

    https://github.com/rmtheis/tess-two

    TessBaseAPI baseApi = new TessBaseAPI();

    It will you unfortunately Allication has stopped, but it compile and installed well

  2. Michael Siu says:

    Hi Lahiru,

    Are you running it on Google Glass? Or a mobile device?

    In your case, I will double check the following:

    – Did you follow all the steps to build Tesseract properly?
    – Did you add google glass lib in your project
    – Does your example Tesseract app running properly on your android phone?

    I will start with these item to make sure it’s running properly.

    Happy coding! 🙂

  3. gsbchkteam says:

    Hi, I’m new to Google glass. Would you tell me where I should “Add the following sample code to the project’s main activity”? Do I need to change the path? Thanks

  4. gsbchkteam says:

    Hi, I have tried the whole afternoon, it seems like I couldn’t run the code under 4.4 SDK. Is it because of the SDK upgrade from 4.0.3 to 4.4? Is there any solution? Where I download the glass sneak peek 4.0.3? Thanks.

  5. gsbchkteam says:

    I should say I can run the code in 4.4 by changing the following.
    In CameraDemoLocalService.java
    //import com.google.android.glass.timeline.TimelineManager;

    // TimelineManager tm = TimelineManager.from(context);
    liveCard = new LiveCard(context,cardId);
    // liveCard.setNonSilent(false); // the livecard runs in the “background” only.

    When I ran the program, I could take a picture but it stucked at “Tap to accept”
    In the debug comment, the last call is “handleGestureTap() called”

    I really want to get it running. Either modify the code or if I can download the 4.0.3 for glass. Any suggestion will be great. Thanks.

    • rui says:

      I got the same problem, but after i checked the logCat, i found it’s because the language model is not put in the right DATA_PATH variable, make sure that in google glass, there is a “tessdata/” folder inside the DATA_PATH, and the language model “eng.traineddata” should be put inside tessdata folder. It would be nice if we could first preview then take a picture with right position and distance instead of suddenly save a picture and ocr it.

  6. gsbchkteam says:

    We saw this bug of startActivityForResult for SDK 4.4, is it true? Thanks
    https://code.google.com/p/google-glass-api/issues/detail?id=560

  7. gsbchkteam says:

    We found out the doBindService return false. Is it the SDK 4.4 problem? Thx

  8. Yours Truly says:

    Hey man, thanks for putting this tutorial up. I’m following it and brushing up on my sword fighting while it compiles.
    One question, how does the training data end up in the library? Do I need to manually import it before compilation?

  9. Michael Siu says:

    @Yours Truly
    You don’t need to manual import anything, after compiling, the OCR should work

  10. Michael Siu says:

    @gsbchkteam

    I haven’t run this project after the update, let me get back to you

  11. Michael Siu says:

    @gsbchkteam,

    After the update, some of the code broke, but it works fine with 4.0.3, i got it to work at demo at the BitCoin Hackathon in Toronto

  12. Great tutorial.. But this doesnt work anymore.. The code uses Timeline which is not supported with the latest version.Can you please post the latest code?

  13. Sushil says:

    Hi
    What Changes need to be done in the upgraded version ?
    Can you please Highlight the code.

  14. Nayal says:

    TessBaseAPI baseApi = new TessBaseAPI(); for android will stop the application. any suggestion
    and how to put image as local resource,not from camera or SD car input

  15. amar says:

    Did u get the solution for TimeLineManager package error.or else can u send me the timeline manager class

Leave a Reply