ZOFTINO.COM android and web dev tutorials

Organize Photos by Content Android Example

This post explains how to create in android a feature that lets user organize photos by its content. The feature can be built using Firebase ML kit image labeling API.

Organize Photos by Content Android Example

To show how to create the feature which categorizes photos based on the content present on the picture, let’s develop an example app which allows user to start the process of extracting the primary content type from the limited number of images present in the photos gallery on the phone, store the image path and content type data in the Firestore database.

The main screen displays the content types in recycler view. On clicking a type, app fetches from the database the list of image paths for the selected content type and displays them in recycler view.

android firebase ml kit image labeling api exampleandroid app code organize photos categorize photos by content

Firebase ML Kit Image Labeling API

Since we use Firebase ML kit image labeling API, let’s see how it works. Using Firebase ML kit image labeling API, you can find out what content, for example paper, chair, flower, bird, etc.., present on a given picture.

First get the bitmap of an image which needs to be in upright position. Since image labeling API can recognize content on 480x360 size images, it is better to resize larger images to improve the performance.

Then create FirebaseVisionImage object by passing bitmap to FirebaseVisionImage.fromBitmap() method.

 FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);

Then create FirebaseVisionImageLabeler object by calling getOnDeviceImageLabeler() method on FirebaseVision instance. If you want to use cloud version of the api, you need to call getCloudImageLabeler() method.

FirebaseVisionImageLabeler labeler = FirebaseVision.getInstance()
                .getOnDeviceImageLabeler();

Then call processImage() method on FirebaseVisionImageLabeler object passing FirebaseVisionImage object to it.

Task t = labeler.processImage(image)

Then register listeners to the task. Success listener’s onSuccess method receives list of FirebaseVisionImageLabel objects which can be used to get confidence of the label and label text. 

    t.addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionImageLabel>>() {
        @Override
        public void onSuccess(List<FirebaseVisionImageLabel> labels) {
	    //do somthing
        }
    })
       .addOnFailureListener(new OnFailureListener() {
        @Override
        public void onFailure(@NonNull Exception e) {
           //do somthing
        }
    });

Project Setup

Since the app uses Firebase, Firebase setup needs to be done for the project. For steps, please follow firebase setup instructions.

Then add libraries to gradle build files.

implementation 'androidx.recyclerview:recyclerview:1.0.0'
implementation 'com.google.firebase:firebase-core:17.0.0'
implementation 'com.google.firebase:firebase-ml-vision:22.0.0'
implementation 'com.google.firebase:firebase-ml-vision-image-label-model:18.0.0'
implementation 'com.google.android.material:material:1.0.0-alpha1'
implementation 'com.google.firebase:firebase-firestore:20.2.0'

Organize Photos by Content Android Example Code

The example has two activities. Main activity provides the option for users to start categorizing photos in the gallery and store the processed data in database. If photos are already categorized and data is available, then the main activity displays the list of content types in recycler view allowing users to click a particular content type to view photos in that type.

Second activity gets the content type that user selected, fetches data from database and displays images in recycler view.

Main Activity

It uses utility class to label images. It contains code to get list of content types for which pictures are available and display them in recycler view.

import android.content.pm.PackageManager;
import android.os.Bundle;
import android.util.Log;
import android.view.View;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import androidx.recyclerview.widget.DividerItemDecoration;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;

import com.google.android.gms.tasks.OnCompleteListener;
import com.google.android.gms.tasks.Task;
import com.google.firebase.firestore.FirebaseFirestore;
import com.google.firebase.firestore.QuerySnapshot;
import com.zoftino.imagecontentfilter.background.ImageContentLabelManager;
import com.zoftino.imagecontentfilter.background.ImageContentLabelTask;

import java.util.List;

public class MainActivity extends AppCompatActivity {

    private static final int PHOTOS_LIMIT = 100;

    private static final String READ_PERM =
            "android.permission.READ_EXTERNAL_STORAGE";

    private RecyclerView rv;
    private FirebaseFirestore firestoreDB;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        //permission to get photos
        if (ContextCompat.checkSelfPermission(this, READ_PERM)
                != PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this,
                    new String[]{READ_PERM}, 2);
        }

        setContentView(R.layout.activity_main);

        firestoreDB = FirebaseFirestore.getInstance();

        rv = findViewById(R.id.content_type_rv);

        LinearLayoutManager recyclerLayoutManager =
                new LinearLayoutManager(this);
        rv.setLayoutManager(recyclerLayoutManager);

        DividerItemDecoration dividerItemDecoration =
                new DividerItemDecoration(this,
                        recyclerLayoutManager.getOrientation());
        dividerItemDecoration.setDrawable(getResources()
                .getDrawable(android.R.drawable.divider_horizontal_bright, null));
        rv.addItemDecoration(dividerItemDecoration);

        //get image categories from firestore database
        //display image categories in recyclerview
        getImageCategories();
    }

    //process photos from gallery
    //categorize them baesd on the content type
    //store content type data in firestore db
    public void categorizePhotos(View v) {
        List<String> imagesUrls = ImageLabelUtil.getCameraImages(this, PHOTOS_LIMIT);

        for (String imgUrl : imagesUrls) {
            ImageContentLabelTask imageLabelTask = new ImageContentLabelTask(imgUrl);
            ImageContentLabelManager.getImageContentManager().labelImageContent(imageLabelTask);
        }
    }

    //get photo content types from db
    //already processed data
    private void getImageCategories() {
        firestoreDB.collection("PhotosCat")
                .get()
                .addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                    @Override
                    public void onComplete(@NonNull Task<QuerySnapshot> task) {
                        if (task.isSuccessful()) {
                            if (task.isSuccessful()) {
                                List<ImageLabel> typesList =
                                        task.getResult().toObjects(ImageLabel.class);
                                //display content types in recycler view
                                PhotoContentTypeRVAdapter contentAdapter = new
                                        PhotoContentTypeRVAdapter(typesList, MainActivity.this);
                                rv.setAdapter(contentAdapter);

                            } else {
                                Log.d("Get Content Types", "Error getting content types");
                            }

                        } else {
                            Log.d("Get Content Types", "Error getting content types");
                        }
                    }
                });
    }
}

Main Activity Layout

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:layout_margin="8dp"
    tools:context=".MainActivity">
    <Button
        android:id="@+id/add_prd"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="8dp"
        android:text="Label 100 Photos from Gallery"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        android:onClick="categorizePhotos"
        style="@style/Widget.AppCompat.Button.Colored"/>

    <TextView
        android:id="@+id/choose_txt"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="8dp"
        android:text="Click Content Type to View Photos"
        android:textAppearance="@style/TextAppearance.AppCompat.Headline"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/add_prd" />

    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/content_type_rv"
        android:scrollbars="vertical"
        android:layout_width="match_parent"
        android:layout_height="500dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/choose_txt" />

</androidx.constraintlayout.widget.ConstraintLayout>

Content Types RecyclerView Adapter

Recycler view adapter for displaying image content types and starting view image activity.

import android.content.Context;
import android.content.Intent;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.TextView;

import androidx.recyclerview.widget.RecyclerView;

import java.util.List;

public class PhotoContentTypeRVAdapter extends
        RecyclerView.Adapter<PhotoContentTypeRVAdapter.ViewHolder> {

    private List<ImageLabel> typeList;
    private Context context;

    public PhotoContentTypeRVAdapter(List<ImageLabel> list, Context ctx) {
        typeList = list;
        context = ctx;
    }

    @Override
    public int getItemCount() {
        return typeList.size();
    }

    @Override
    public PhotoContentTypeRVAdapter.ViewHolder
    onCreateViewHolder(ViewGroup parent, int viewType) {

        View view = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.photo_content_type_item, parent, false);

        PhotoContentTypeRVAdapter.ViewHolder viewHolder =
                new PhotoContentTypeRVAdapter.ViewHolder(view);
        return viewHolder;
    }

    @Override
    public void onBindViewHolder(PhotoContentTypeRVAdapter.ViewHolder holder, int position) {

        final ImageLabel type = typeList.get(position);

        holder.textView.setText(type.getImageContentType());

        holder.textView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                //get the selected content type and pass it to activity
                getPicturePathsByContentType(type.getImageContentType());
            }
        });
    }

    public class ViewHolder extends RecyclerView.ViewHolder {
        public TextView textView;

        public ViewHolder(View view) {
            super(view);
            textView = view.findViewById(R.id.photo_content_type);
        }
    }

    //start activity which displays photos in recycler view
    private void getPicturePathsByContentType(String type){
        Intent i = new Intent();
        i.putExtra("contentType", type);
        i.setClass(context, PhotosByContentActivity.class);
        context.startActivity(i);
    }
}

Content Types RecyclerView Item Layout

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:orientation="vertical">
    <TextView
        android:id="@+id/photo_content_type"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_margin="16dp"
        android:textAppearance="@style/TextAppearance.AppCompat.Headline"/>
</LinearLayout>

View Photos Activity

Activity gets the selected content type from the intent, gets the list of image paths from database, resizes them and displays in recycler view.

package com.zoftino.imagecontentfilter;

import android.os.Bundle;
import android.util.Log;
import android.widget.TextView;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.recyclerview.widget.DividerItemDecoration;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;

import com.google.android.gms.tasks.OnCompleteListener;
import com.google.android.gms.tasks.Task;
import com.google.firebase.firestore.FirebaseFirestore;
import com.google.firebase.firestore.QuerySnapshot;

import java.util.List;

public class PhotosByContentActivity extends AppCompatActivity {

    private RecyclerView rv;
    private FirebaseFirestore firestoreDB;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_photos);

        firestoreDB = FirebaseFirestore.getInstance();
        rv = findViewById(R.id.photos_rv);

        LinearLayoutManager recyclerLayoutManager =
                new LinearLayoutManager(this);
        rv.setLayoutManager(recyclerLayoutManager);

        DividerItemDecoration dividerItemDecoration =
                new DividerItemDecoration(this,
                        recyclerLayoutManager.getOrientation());
        dividerItemDecoration.setDrawable(getResources()
                .getDrawable(android.R.drawable.divider_horizontal_dim_dark, null));
        rv.addItemDecoration(dividerItemDecoration);

        //get user selected content type from the intent
        String contentType = getIntent().getStringExtra("contentType");

        TextView tv = findViewById(R.id.photos_txt);
        tv.setText(contentType+" Photos from Gallry");

        //display list of photos from the selected type in recycler view
        getImageInfo(contentType);
    }
    //get photo paths from database for the selected content type
    private  void getImageInfo(String imageLabel) {
        firestoreDB.collection("PhotosInfo").document("myPics")
                .collection(imageLabel)
                .get()
                .addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                    @Override
                    public void onComplete(@NonNull Task<QuerySnapshot> task) {
                        if (task.isSuccessful()) {

                            List<ImageLabel> imagesInfo = task.getResult()
                                    .toObjects(ImageLabel.class);

                            //display the selected photos in recycler view
                            PhotosRecyclerViewAdapter photoAdapter = new
                                    PhotosRecyclerViewAdapter(imagesInfo,
                                    PhotosByContentActivity.this);
                            rv.setAdapter(photoAdapter);
                        } else {
                            Log.d("Get photos by type", "Error getting photos info");
                        }
                    }
                });
    }
}

View Photos Activity Layout

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">
    <TextView
        android:id="@+id/photos_txt"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_margin="8dp"
        android:text="Photos"
        android:textAppearance="@style/TextAppearance.AppCompat.Headline"/>
    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/photos_rv"
        android:scrollbars="vertical"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>
</LinearLayout>

View Photos RecyclerView Adapter

import android.content.Context;
import android.graphics.Bitmap;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.ImageView;

import androidx.recyclerview.widget.RecyclerView;

import java.util.List;

public class PhotosRecyclerViewAdapter extends
        RecyclerView.Adapter<PhotosRecyclerViewAdapter.ViewHolder> {

    private List<ImageLabel> imageLst;
    private Context context;

    public PhotosRecyclerViewAdapter(List<ImageLabel> list, Context ctx) {
        imageLst = list;
        context = ctx;
    }

    @Override
    public int getItemCount() {
        return imageLst.size();
    }

    @Override
    public PhotosRecyclerViewAdapter.ViewHolder
    onCreateViewHolder(ViewGroup parent, int viewType) {

        View view = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.photo_item, parent, false);

        PhotosRecyclerViewAdapter.ViewHolder viewHolder =
                new PhotosRecyclerViewAdapter.ViewHolder(view);
        return viewHolder;
    }

    @Override
    public void onBindViewHolder(PhotosRecyclerViewAdapter.ViewHolder holder, int position) {

        final ImageLabel imagePath = imageLst.get(position);
        holder.image.setImageBitmap(processImageForDisplay(imagePath.getImagePath()));

    }

    public class ViewHolder extends RecyclerView.ViewHolder {
        public ImageView image;

        public ViewHolder(View view) {
            super(view);
            image = view.findViewById(R.id.photo);
        }
    }
    //resize image and display as recycler view item
    public Bitmap processImageForDisplay(String imagUrl){
        Bitmap bmp = ImageLabelUtil.getUprightImage(imagUrl);
        return ImageLabelUtil.resizeImage(bmp, context);
    }
}

View Photos RecyclerView Item Layout

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:orientation="vertical">
    <ImageView
        android:id="@+id/photo"
        android:layout_width="match_parent"
        android:layout_height="200dp"
        android:layout_margin="8dp"/>
</LinearLayout>

Utility Class

This class contains code to resize and upright images, label images and save data in database.

import android.app.Activity;
import android.content.Context;
import android.database.Cursor;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Matrix;
import android.media.ExifInterface;
import android.os.Environment;
import android.provider.MediaStore;
import android.util.DisplayMetrics;
import android.util.Log;

import androidx.annotation.NonNull;

import com.google.android.gms.tasks.OnFailureListener;
import com.google.android.gms.tasks.OnSuccessListener;
import com.google.android.gms.tasks.Task;
import com.google.firebase.firestore.DocumentReference;
import com.google.firebase.firestore.FirebaseFirestore;
import com.google.firebase.ml.vision.FirebaseVision;
import com.google.firebase.ml.vision.common.FirebaseVisionImage;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabel;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabeler;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class ImageLabelUtil {

    private static int IMAGE_HEIGHT = 360;
    private static int IMAGE_WIDTH = 480;
    private static FirebaseFirestore firestoreDB = FirebaseFirestore.getInstance();

    //get paths of images from the gallery,
    //restrict the number of item to the limit given
    public static List<String> getCameraImages(Context context, int limit) {
        final String CAMERA_IMAGES = Environment
                .getExternalStorageDirectory().toString() + "/DCIM/Camera";

        final String CAMERA_IMAGES_ID = String.valueOf(
                CAMERA_IMAGES.toLowerCase().hashCode());

        final String[] projection = {MediaStore.Images.Media.DATA};
        final String selection = MediaStore.Images.Media.BUCKET_ID + " = ?";
        final String[] selectionArgs = {CAMERA_IMAGES_ID};
        final Cursor cursor = context.getContentResolver()
                .query(MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
                        projection,
                        selection,
                        selectionArgs,
                        null);
        ArrayList<String> result = new ArrayList<String>(cursor.getCount());
        if (cursor.moveToFirst()) {
            final int dataColumn =
                    cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATA);
            do {
                final String data = cursor.getString(dataColumn);
                result.add(data);
            } while (cursor.moveToNext() && cursor.getPosition() < limit);
        }
        cursor.close();
        return result;
    }

    public static Bitmap resizeImage(Bitmap bitmap) {
        return Bitmap.createScaledBitmap(bitmap, IMAGE_WIDTH, IMAGE_HEIGHT, true);
    }

    public static Bitmap resizeImage(Bitmap bitmap, Context ctx){
        DisplayMetrics displayMetrics = new DisplayMetrics();
        ((Activity)ctx).getWindowManager()
                .getDefaultDisplay()
                .getMetrics(displayMetrics);
        int width = displayMetrics.widthPixels;

        return Bitmap.createScaledBitmap(bitmap, width, width, true);
    }

    public static Bitmap getUprightImage(String imgUrl) {

        ExifInterface exif = null;
        try {
            exif = new ExifInterface(imgUrl);
        } catch (IOException e) {
        }

        int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, 1);
        int rotation = 0;
        switch (orientation) {
            case 3:
                rotation = 180;
                break;
            case 6:
                rotation = 90;
                break;
            case 8:
                rotation = 270;
                break;
        }
        Matrix matrix = new Matrix();
        matrix.postRotate(rotation);

        Bitmap bitmap = BitmapFactory.decodeFile(imgUrl);
        //rotate image
        bitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(),
                bitmap.getHeight(), matrix, true);
        return bitmap;
    }

    //For the given image, find the  primary content type
    //using firebase ml kit image labeling api
    public static void labelImage(String imgUrl) {
        Bitmap bitmap = getUprightImage(imgUrl);
        bitmap = resizeImage(bitmap);

        getContentTypePicture(bitmap, imgUrl);
    }

    public static void getContentTypePicture(Bitmap bitmap, final String path) {
        FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
        FirebaseVisionImageLabeler labeler = FirebaseVision.getInstance()
                .getOnDeviceImageLabeler();
        Task r = labeler.processImage(image)
                .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionImageLabel>>() {
                    @Override
                    public void onSuccess(List<FirebaseVisionImageLabel> labels) {
                        String text = "";
                        float confidence = 0;
                        float highConfidence = 0;
                        for (FirebaseVisionImageLabel label : labels) {
                            confidence = label.getConfidence();

                            if (highConfidence < confidence) {
                                highConfidence = confidence;
                                text = label.getText();
                            }
                            Log.d("images content info ",
                                    text + " " + confidence);
                        }
                        saveImageContentInfo(path, text);
                    }
                })
                .addOnFailureListener(new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        Log.d("Image Labler", "failed to find photo content type");
                    }
                });
    }

    private static void saveImageContentInfo(String path, String contentType) {
        ImageLabel il = new ImageLabel();
        il.setImagePath(path);
        il.setImageContentType(contentType);

        saveImageInfo(il);
        saveImageCategories(contentType);
    }

    //save image path and content type info in firestore db
    private static void saveImageInfo(ImageLabel imageLabel) {
        firestoreDB.collection("PhotosInfo").document("myPics")
                .collection(imageLabel.getImageContentType())
                .add(imageLabel)
                .addOnSuccessListener(new OnSuccessListener<DocumentReference>() {
                    @Override
                    public void onSuccess(DocumentReference documentReference) {
                        Log.d("Image Lable", "add photo content info to database");
                    }
                })
                .addOnFailureListener(new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        Log.d("Image Labler", "failed to add photo content info to db");
                    }
                });
    }
    //save content type in db
    private static void saveImageCategories(String imageCat) {
        ImageLabel il = new ImageLabel();
                il.setImageContentType(imageCat);
        firestoreDB.collection("PhotosCat").document(imageCat).set(il)
                .addOnSuccessListener(new OnSuccessListener<Void>(){
                    @Override
                    public void onSuccess(Void v) {

                    }
                })
                .addOnFailureListener(new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        Log.d("Image Labler", "failed to add photo category to db");
                    }
                });
    }
}

POJO

public class ImageLabel {
    private String imagePath;
    private String imageContentType;

    public String getImagePath() {
        return imagePath;
    }

    public void setImagePath(String imagePath) {
        this.imagePath = imagePath;
    }

    public String getImageContentType() {
        return imageContentType;
    }

    public void setImageContentType(String imageContentType) {
        this.imageContentType = imageContentType;
    }
}

About

Android app development tutorials and web app development tutorials with programming examples and code samples.

Recommended

Android MultiAutoCompleteTextView Tutorial

Android TextSwitcher Tutorial

Android ImageSwitcher Tutorial

Android ViewFlipper Tutorial

Android AdapterViewFlipper Example

Android StackView Tutorial

Android Chronometer Timer Stopwatch Tutorial

Android VideoView MediaController Playing Videos Tutorial

Android VideoView Playing Videos Full Screen

Android Image Full Screen Example

Android Search Dialog with Search Suggestions Example

Android MatrixCursor Example

Android Search Functionality Using SearchView & Room

Android SearchView Dropdown Suggestions

Android Contacts Search Using SearchView

Android ImageView Tutorial

Android WebView Tutorial

Android Google Maps Tutorial

Google Maps Android Custom Info Window Example

Android MapView Tutorial