Intermediate: Find the Image Decode and Encode in Harmony OS

Overview

In this article, I will create a demo app along with the integration of Image Encode and Decode APIs which is based on Harmony OS. I will provide the use case of Image encode and PixalMap Editing in Harmony OS based on application.

Harmony OS Security Introduction

Harmony OS offers APIs to develop image-related functions, including image decoding, encoding, basic pixel map operations, and image editing. User can combine the APIs to implement complex image processing.

1. Image decoding is to convert images in different archive formats (such as JPEG and PNG) to uncompressed pixel maps that can be processed in applications or systems.
2. Pixel map is an uncompressed bitmap after image decoding. It is used for image display or further processing.
3. Incremental decoding is for a scenario where complete image data cannot be provided at a time. The data is incremented and decoded for several times till the image decoding is complete.
4. Pre-multiplication is the process of multiplying the value of each RGB channel by the opaque ratio (ranging from 0 to 1) of the alpha channel. This facilitates subsequent synthesis and overlay. Without pre-multiplication, the value of each RGB channel is the original value of the image, which is irrelevant to the alpha channel.
5. Image encoding is to encode uncompressed pixel maps and converts them to different archive formats (such as JPEG and PNG), which facilitates image processing in applications or systems.

API Overview

Decoding Images

Create an ImageSource object and use SourceOptions to specify the format of the source image. The format information is only used as a prompt for the decoder. Correct information helps to improve the decoding efficiency. If the format information is not set or is incorrect, the system automatically detects the source image format. If you do not need SourceOptions, set it to null when you call the create method.

​ImageSource.SourceOptions srcOpts = new ImageSource.SourceOptions();
srcOpts.formatHint = "image/png";
String pathName = "/sdcard/image.png";
ImageSource imageSource = ImageSource.create(pathName, srcOpts);
// Set SourceOptions to null when calling the create method.
ImageSource imageSourceNoOptions = ImageSource.create(pathName, null);
​Copy code

Set decoding parameters and decode the source image to obtain the pixel map. Image processing is supported during decoding.

Set desiredSize to specify target size after scaling. If the values are all set to 0, scaling will not be performed.

Set desiredRegion to specify the target rectangular area after cropping. If the values are all set to 0, cropping will not be performed.

Set rotateDegrees to specify the rotation angle. The image will be rotated clockwise at the center.

If you do not need DecodingOptions, set it to null when you call the createPixelMap method.

// Common decoding with scaling, cropping, and rotation
ImageSource.DecodingOptions decodingOpts = new ImageSource.DecodingOptions();
decodingOpts.desiredSize = new Size(100, 2000);
decodingOpts.desiredRegion = new Rect(0, 0, 100, 100);
decodingOpts.rotateDegrees = 90;
PixelMap pixelMap = imageSource.createPixelmap(decodingOpts);
// Common decoding
PixelMap pixelMapNoOptions = imageSource.createPixelmap(null);Copy code

Image Property Decoding

Create an ImageSource object and use SourceOptions to specify the format of the source image. The format information is only used as a prompt for the decoder. Correct information helps to improve the decoding efficiency. If the format information is not set or is incorrect, the system automatically detects the source image format.

ImageSource.SourceOptions srcOpts = new ImageSource.SourceOptions();
srcOpts.formatHint = "image/jpeg";
String pathName = "/sdcard/image.jpg";
ImageSource imageSource = ImageSource.create(pathName, srcOpts);
Copy code

Obtain thumbnail information.

int format = imageSource.getThumbnailFormat();
byte[] thumbnailBytes = imageSource.getImageThumbnailBytes();
// Decode the thumbnail and convert it to a PixelMap object.
ImageSource.DecodingOptions decodingOpts = new ImageSource.DecodingOptions();
PixelMap thumbnailPixelmap = imageSource.createThumbnailPixelmap(decodingOpts, false);Copy code

PixalMap Editing

Create a PixelMap object.

// Set a pixel color array and create a pixel map from the array.
int[] defaultColors = new int[] {5, 5, 5, 5, 6, 6, 3, 3, 3, 0};
PixelMap.InitializationOptions initializationOptions = new PixelMap.InitializationOptions();
initializationOptions.size = new Size(3, 2);
initializationOptions.pixelFormat = PixelFormat.ARGB_8888;
initializationOptions.editable = true;
PixelMap pixelMap1 = PixelMap.create(defaultColors, initializationOptions);

// Specify the initialization options for the creation.
PixelMap pixelMap2 = PixelMap.create(initializationOptions);

// Create a pixel map from the data source, which is another pixel map.
PixelMap pixelMap3 = PixelMap.create(pixelMap2, initializationOptions);
Obtain information from the PixelMap object.
long capacity = pixelMap.getPixelBytesCapacity();
long bytesNumber = pixelMap.getPixelBytesNumber();
int rowBytes = pixelMap.getBytesNumberPerRow();
byte[] ninePatchData = pixelMap.getNinePatchChunk();Copy code

Read and write pixel data of a pixel map.

// Read pixel data at a specified position.
int color = pixelMap.readPixel(new Position(1, 1));

// Read pixel data from a specified region.
int[] pixelArray = new int[50];
Rect region = new Rect(0, 0, 10, 5);
pixelMap.readPixels(pixelArray, 0, 10, region);

// Read pixel data to the buffer.
IntBuffer pixelBuf = IntBuffer.allocate(50);
pixelMap.readPixels(pixelBuf);

// Write pixel data at the specified position.
pixelMap.writePixel(new Position(1, 1), 0xFF112233);

// Write pixel data to the specified region.
pixelMap.writePixels(pixelArray, 0, 10, region);
// Write pixel data into the buffer.
pixelMap.writePixels(intBuf);Copy code

Prerequisite

1. Harmony OS phone.
2. Java JDK.
3. DevEco Studio.

App Development

Create a New Harmony OS Project.

Configure Project config.json.

Configure Project Gradle.

Configure App Gradle.

Create Ability class with XML UI.

MainAbilitySlice.java:

This ability performs all the operation of Image Decode and encode.

package com.hos.imagedemo.slice;
import com.hos.imagedemo.ResourceTable;
import ohos.aafwk.ability.AbilitySlice;
import ohos.aafwk.content.Intent;
import ohos.aafwk.ability.AbilitySlice;
import ohos.aafwk.content.Intent;
import ohos.agp.components.Component;
import ohos.agp.components.Image;
import ohos.agp.components.Text;
import ohos.agp.utils.Color;
import ohos.global.resource.RawFileEntry;
import ohos.global.resource.Resource;
import ohos.hiviewdfx.HiLog;
import ohos.hiviewdfx.HiLogLabel;
import ohos.media.image.ImagePacker;
import ohos.media.image.ImageSource;
import ohos.media.image.PixelMap;
import ohos.media.image.common.PixelFormat;
import ohos.media.image.common.Position;
import ohos.media.image.common.PropertyKey;
import ohos.media.image.common.Rect;
import ohos.media.image.common.Size;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Arrays;
public class MainAbilitySlice extends AbilitySlice {
    private static final String TAG = MainAbilitySlice.class.getSimpleName();
    private static final HiLogLabel LABEL_LOG = new HiLogLabel(3, 0xD000F00, TAG);
    private static final int CACHE_SIZE = 1024;
    private static final String RAW_IMAGE_PATH = "entry/resources/rawfile/test.png";
    private static final String RAW_IMAGE_PATH2 = "entry/resources/rawfile/test.jpg";
    private Image showFirstImage;
    private Image showSecondImage;
    private Text showResultText;
    private String pngCachePath;
    private String jpgCachePath;
    private String encodeOutPath;
    @Override
    public void onStart(Intent intent) {
        super.onStart(intent);
        super.setUIContent(ResourceTable.Layout_ability_main);
        initComponents();
        initData();
    }
    private void initData() {
        pngCachePath = new File(getFilesDir(), "test.png").getPath();
        jpgCachePath = new File(getFilesDir(), "test.jpg").getPath();
        encodeOutPath = new File(getFilesDir(), "test_encode.jpg").getPath();
        writeToDisk(RAW_IMAGE_PATH, pngCachePath);
        writeToDisk(RAW_IMAGE_PATH2, jpgCachePath);
    }
    private void initComponents() {
        Component commonDecodeButton = findComponentById(ResourceTable.Id_common_decode_button);
        Component regionDecodeButton = findComponentById(ResourceTable.Id_region_decode_button);
        Component encodeButton = findComponentById(ResourceTable.Id_encode_button);
        Component editButton = findComponentById(ResourceTable.Id_edit_button);
        commonDecodeButton.setClickedListener(this::commonDecode);
        regionDecodeButton.setClickedListener(this::regionDecode);
        encodeButton.setClickedListener(this::encode);
        editButton.setClickedListener(this::edit);
        Component attributeButton = findComponentById(ResourceTable.Id_altitude_button);
        attributeButton.setClickedListener(this::attribute);
        showResultText = (Text) findComponentById(ResourceTable.Id_result_text);
        showFirstImage = (Image) findComponentById(ResourceTable.Id_test_image1);
        showSecondImage = (Image) findComponentById(ResourceTable.Id_test_image2);
    }
    private void commonDecode(Component component) {
        cleanComponents();
        ImageSource.SourceOptions srcOpts = new ImageSource.SourceOptions();
        srcOpts.formatHint = "image/png";
        String pathName = pngCachePath;
        ImageSource imageSource = ImageSource.create(pathName, srcOpts);
        PixelMap pixelMapNoOptions = imageSource.createPixelmap(null);
        showFirstImage.setPixelMap(pixelMapNoOptions);
        ImageSource.DecodingOptions decodingOpts = new ImageSource.DecodingOptions();
        decodingOpts.desiredSize = new Size(600, 300);
        decodingOpts.desiredRegion = new Rect(0, 0, 300, 150);
        PixelMap pixelMap = imageSource.createPixelmap(decodingOpts);
        showSecondImage.setPixelMap(pixelMap);
        imageSource.release();
        pixelMapNoOptions.release();
    }
    private void regionDecode(Component component) {
        cleanComponents();
        ImageSource.SourceOptions srcOpts = new ImageSource.SourceOptions();
        srcOpts.formatHint = "image/jpeg";
        ImageSource.IncrementalSourceOptions incOpts = new ImageSource.IncrementalSourceOptions();
        incOpts.opts = srcOpts;
        incOpts.mode = ImageSource.UpdateMode.INCREMENTAL_DATA;
        ImageSource imageSource = ImageSource.createIncrementalSource(incOpts);
        RawFileEntry rawFileEntry = getResourceManager().getRawFileEntry(RAW_IMAGE_PATH);
        try (Resource resource = rawFileEntry.openRawFile()) {
            byte[] cache = new byte[CACHE_SIZE];
            int len = resource.read(cache);
            while (len != -1) {
                imageSource.updateData(cache, 0, len, false);
                if (len < CACHE_SIZE) {
                    imageSource.updateData(cache, 0, len, true);
                    ImageSource.DecodingOptions decodingOpts2 = new ImageSource.DecodingOptions();
                    PixelMap pixelmap = imageSource.createPixelmap(decodingOpts2);
                    showSecondImage.setPixelMap(pixelmap);
                    pixelmap.release();
                }
                len = resource.read(cache);
            }
        } catch (IOException e) {
            HiLog.info(LABEL_LOG, "%{public}s", "regionDecode IOException ");
        }
        imageSource.release();
    }
    private void encode(Component component) {
        cleanComponents();
        ImagePacker imagePacker = ImagePacker.create();
        ImagePacker.PackingOptions packingOptions = new ImagePacker.PackingOptions();
        packingOptions.quality = 90;
        try (FileOutputStream outputStream = new FileOutputStream(encodeOutPath)) {
            imagePacker.initializePacking(outputStream, packingOptions);
            ImageSource imageSource = ImageSource.create(pngCachePath, null);
            PixelMap pixelMap = imageSource.createPixelmap(null);
            boolean result = imagePacker.addImage(pixelMap);
            showResultText.setText(
                    "Encode result : " + result + System.lineSeparator() + "OutputFilePath：" + encodeOutPath);
            imageSource.release();
            pixelMap.release();
        } catch (IOException e) {
            HiLog.info(LABEL_LOG, "%{public}s", "encode IOException ");
        }
        imagePacker.release();
    }
    private void attribute(Component component) {
        cleanComponents();
        ImageSource.SourceOptions srcOpts = new ImageSource.SourceOptions();
        srcOpts.formatHint = "image/jpeg";
        ImageSource imageSource = ImageSource.create(jpgCachePath, srcOpts);
        int format = imageSource.getThumbnailFormat();
        byte[] thumbnailBytes = imageSource.getImageThumbnailBytes();
        ImageSource.DecodingOptions decodingOpts = new ImageSource.DecodingOptions();
        PixelMap thumbnailPixelMap = imageSource.createThumbnailPixelmap(decodingOpts, false);
        String location = imageSource.getImagePropertyString(PropertyKey.Exif.SUBJECT_LOCATION);
        HiLog.info(LABEL_LOG, "%{public}s", "imageExif location : " + location);
        showResultText.setText("ImageSource attribute : createThumbnailPixelMap");
        showSecondImage.setPixelMap(thumbnailPixelMap);
        imageSource.release();
        thumbnailPixelMap.release();
    }
    private void edit(Component component) {
        cleanComponents();
        int colorsWidth = 600;
        int colorsHeight = 300;
        PixelMap.InitializationOptions initializationOptions = new PixelMap.InitializationOptions();
        initializationOptions.size = new Size(colorsWidth, colorsHeight);
        initializationOptions.pixelFormat = PixelFormat.ARGB_8888;
        initializationOptions.editable = true;
        int[] colors = new int[colorsWidth * colorsHeight];
        Arrays.fill(colors, Color.RED.getValue());
        PixelMap pixelMap = PixelMap.create(colors, initializationOptions);
        showFirstImage.setPixelMap(pixelMap);
        PixelMap pixelMap2 = PixelMap.create(pixelMap, initializationOptions);
        int color = pixelMap2.readPixel(new Position(1, 1));
        HiLog.info(LABEL_LOG, "%{public}s", "pixelMapEdit readPixel color ：" + color);
        pixelMap2.writePixel(new Position(100, 100), Color.BLACK.getValue());
        pixelMap2.writePixel(new Position(100, 101), Color.BLACK.getValue());
        pixelMap2.writePixel(new Position(101, 100), Color.BLACK.getValue());
        pixelMap2.writePixel(new Position(101, 101), Color.BLACK.getValue());
        int[] pixelArray = new int[500];
        Arrays.fill(pixelArray, Color.BLACK.getValue());
        Rect region = new Rect(0, 0, 20, 10);
        pixelMap2.writePixels(pixelArray, 0, 20, region);
        showSecondImage.setPixelMap(pixelMap2);
        long capacity = pixelMap.getPixelBytesCapacity();
        long bytesNumber = pixelMap.getPixelBytesNumber();
        int rowBytes = pixelMap.getBytesNumberPerRow();
        byte[] ninePatchData = pixelMap.getNinePatchChunk();
        showResultText.setText(
                "This pixelMap detail info :" + System.lineSeparator() + "capacity = " + capacity + System.lineSeparator()
                        + "bytesNumber = " + bytesNumber + System.lineSeparator() + "rowBytes = " + rowBytes
                        + System.lineSeparator() + "ninePatchData = " + ninePatchData + System.lineSeparator());
        pixelMap.release();
        pixelMap2.release();
    }
    private void cleanComponents() {
        showResultText.setText("");
        showFirstImage.setPixelMap(null);
        showSecondImage.setPixelMap(null);
    }
    private void writeToDisk(String rawFilePathString, String targetFilePath) {
        File file = new File(targetFilePath);
        if (file.exists()) {
            return;
        }
        RawFileEntry rawFileEntry = getResourceManager().getRawFileEntry(rawFilePathString);
        try (FileOutputStream output = new FileOutputStream(new File(targetFilePath))) {
            Resource resource = rawFileEntry.openRawFile();
            byte[] cache = new byte[CACHE_SIZE];
            int len = resource.read(cache);
            while (len != -1) {
                output.write(cache, 0, len);
                len = resource.read(cache);
            }
        } catch (IOException e) {
            HiLog.info(LABEL_LOG, "%{public}s", "writeEntryToFile IOException ");
        }
    }
}

MainAbility.java:

package com.hos.imagedemo;
import com.hos.imagedemo.slice.MainAbilitySlice;
import ohos.aafwk.ability.Ability;
import ohos.aafwk.content.Intent;
public class MainAbility extends Ability {
    @Override
    public void onStart(Intent intent) {
        super.onStart(intent);
        super.setMainRoute(MainAbilitySlice.class.getName());
    }
}

ability_main.xml:

<?xml version="1.0" encoding="utf-8"?>
<DirectionalLayout
    xmlns:ohos="http://schemas.huawei.com/res/ohos"
    ohos:height="match_parent"
    ohos:width="match_parent"
    ohos:alignment="horizontal_center"
    ohos:orientation="vertical"
    ohos:background_element="$media:ic_oho"
    ohos:padding="5vp">
    <Button
        ohos:id="$+id:common_decode_button"
        ohos:height="match_content"
        ohos:width="300vp"
        ohos:background_element="$graphic:button_bg"
        ohos:padding="5vp"
        ohos:text="Common Decode"
        ohos:text_alignment="center"
        ohos:text_size="18fp"
        ohos:top_margin="50vp"/>
    <Button
        ohos:id="$+id:region_decode_button"
        ohos:height="match_content"
        ohos:width="300vp"
        ohos:background_element="$graphic:button_bg"
        ohos:padding="5vp"
        ohos:text="Region Decode"
        ohos:text_alignment="center"
        ohos:text_size="18fp"
        ohos:top_margin="10vp"/>
    <Button
        ohos:id="$+id:encode_button"
        ohos:height="match_content"
        ohos:width="300vp"
        ohos:background_element="$graphic:button_bg"
        ohos:padding="5vp"
        ohos:text="Encode"
        ohos:text_alignment="center"
        ohos:text_size="18fp"
        ohos:top_margin="10vp"/>
    <Button
        ohos:id="$+id:edit_button"
        ohos:height="match_content"
        ohos:width="300vp"
        ohos:background_element="$graphic:button_bg"
        ohos:padding="5vp"
        ohos:text="PixelMap Edit"
        ohos:text_alignment="center"
        ohos:text_size="18fp"
        ohos:top_margin="10vp"/>
    <Button
        ohos:id="$+id:altitude_button"
        ohos:height="match_content"
        ohos:width="300vp"
        ohos:background_element="$graphic:button_bg"
        ohos:padding="5vp"
        ohos:text="PixelMap Attribute"
        ohos:text_alignment="center"
        ohos:text_size="18fp"
        ohos:top_margin="10vp"/>
    <Image
        ohos:id="$+id:test_image1"
        ohos:height="100vp"
        ohos:width="match_content"
        ohos:scale_mode="zoom_center"
        ohos:top_margin="20vp"/>
    <Text
        ohos:id="$+id:result_text"
        ohos:height="match_content"
        ohos:width="match_parent"
        ohos:multiple_lines="true"
        ohos:padding="10vp"
        ohos:text_alignment="center"
        ohos:text_size="16fp"/>
    <Image
        ohos:id="$+id:test_image2"
        ohos:height="100vp"
        ohos:width="match_content"
        ohos:margin="5vp"
        ohos:scale_mode="zoom_center"/>
</DirectionalLayout>

App Build Result

Tips and Tricks

1. You need to implement image decoding for your application to convert any archived image of a supported format to a pixel map for displaying and other image processing operations, such as rotation, scaling, and cropping. JPEG, PNG, GIF, HEIF, WebP and BMP are supported for image decoding.
2. You can use APIs to encode pixel maps and convert them to images in different archive formats for subsequent processing, such as storage and transmission. Currently, only the JPEG format is supported for image encoding.
3. You can use APIs to obtain property information contained in an image, such as exchangeable image file format (Exif) properties.

Conclusion

In this article, we have learned how to implement Image Decoding in Harmony OS application. In this application, I have explained that how user can decode encode JPEG format based images.

Thanks for reading this article. Be sure to like and comments to this article, if you found it helpful. It means a lot to me.

References

Harmony OS Doc Link:

https://developer.harmonyos.com/en/docs/documentation/doc-guides/media-image-overview-0000000000031765