Intermediate: Integration of Media Data Management APIs in Harmony OS
Overview
In this article, I will create a demo app along with the integration of Media Data Management APIs which is based on Harmony OS. I will provide the use case of Video based media file which gets metadata in Harmony OS based on application.
Harmony OS Media Data Management
HarmonyOS media data management module supports the development of features, such as obtaining media metadata and frame data.
Pixel map
Pixel map is an uncompressed bitmap after image decoding. It is used for image display or further processing.
Media metadata
Media metadata is used to describe media information. For example: a media item title and duration.
Limitations and Constraints
Call the release() method to release native resources in a timely manner, if the AVMetadataHelper object will not be used anymore.
API Overview
Obtaining Frame Data
Create an AVMetadataHelper instance. Call setSource to set the media file to read. If the media file is not set or set incorrectly, the subsequent operations cannot be performed.
AVMetadataHelper avMetadataHelper = new AVMetadataHelper();avMetadataHelper.setSource("/path/short_video.mp4");
Specify the frame query option and timestamp in the video source for obtaining frame data.
PixelMap pixelMap = avMetadataHelper.fetchVideoPixelMapByTime(1000L, 0x00);After obtaining the PixelMap object and processing information, call release() to release the media resources that has been read.
avMetadataHelper.release();
Obtaining Media Metadata
Create an AVMetadataHelper instance. Call setSource to set the media file to read. If the media file is not set or set incorrectly, the subsequent operations cannot be performed.
AVMetadataHelper avMetadataHelper= new AVMetadataHelper();avMetadataHelper.setSource("/path/short_video.mp4");
Specify the key of the media metadata to obtain. The following sample code is used to obtain the media item duration:
String result = avMetadataHelper.resolveMetadata(AVMetadataHelper.AV_KEY_DURATION);After obtaining the media metadata, call release() to release the media resources that has been read.
avMetadataHelper.release();Obtaining Image Data from an Audio File.
Create an AVMetadataHelper instance. Call setSource to set the radio file to read. If the radio file is not set or set incorrectly, the subsequent operations cannot be performed.
AVMetadataHelper avMetadataHelper= new AVMetadataHelper();avMetadataHelper.setSource("/path/short_video.mp4");
Obtain image data from an audio file.
byte[] data = avMetadataHelper.resolveImage();After obtaining the image data, call release() to release the media resources that has been read.
avMetadataHelper.release();
The predefined key used to obtain the video URI from the external media storage is AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI.
Access media metadata based on the URI provided by the external media storage.
DataAbilityHelper helper = DataAbilityHelper.creator(context);try { DataAbilityPredicates predicates = new DataAbilityPredicates(); // Set query criteria. predicates.equalTo(AVStorage.Video.Media.DATA, "xxxxx"); // If columns is set to null, all columns are queried. In this example, the id field is queried. ResultSet result = helper.query(AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI, new String[]{AVStorage.Video.Media.ID}, predicates); if (result == null) { return; } while (result.goToNextRow()) { result.getInt(result.getColumnIndexForName(AVStorage.Video.Media.ID)); // Obtain the value of the id field. } result.close();} catch (DataAbilityRemoteException e) { // ...}
After obtaining the media item ID, play the media item that is set as the source file.
Uri uri = Uri.appendEncodedPathToUri (AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI, String.valueOf(id)); // The value of parameter id is the ID obtained in Step 1.Player player = new Player(context);DataAbilityHelper helper = DataAbilityHelper.creator(context);player.setSource(new Source(helper.openFile(uri, "r")));
Prerequisite
- Harmony OS phone.
- Java JDK.
- DevEco Studio.
App Development
- Create a New Harmony OS Project.
2. Configure Project config.json.
3. Configure Project Gradle.
4. Configure App Gradle.
5. Create Ability class with XML UI.
MainAbilitySlice.java:
This ability performs all the operation of Media Data Management.
import ohos.aafwk.ability.DataAbilityHelper;
import ohos.aafwk.ability.DataAbilityRemoteException;
import ohos.agp.components.*;
import ohos.app.Context;
import ohos.data.dataability.DataAbilityPredicates;
import ohos.data.resultset.ResultSet;
import ohos.media.common.Source;
import ohos.media.photokit.metadata.AVStorage;
import ohos.media.player.Player;
import ohos.samples.videoplayer.ResourceTable;
import ohos.samples.videoplayer.utils.LogUtil;
import ohos.samples.videoplayer.utils.VideoElementManager;
import ohos.samples.videoplayer.utils.VideoPlayerPlugin;import ohos.aafwk.ability.AbilitySlice;
import ohos.aafwk.content.Intent;
import ohos.agp.components.surfaceprovider.SurfaceProvider;
import ohos.agp.graphics.Surface;
import ohos.agp.graphics.SurfaceOps;
import ohos.media.common.sessioncore.AVElement;
import ohos.utils.net.Uri;import java.util.ArrayList;
import java.util.List;
public class MainAbilitySlice extends AbilitySlice {
private static final String TAG = MainAbilitySlice.class.getSimpleName(); private SurfaceProvider surfaceProvider; private Text titleText; private VideoPlayerPlugin videoPlayerPlugin; private ListContainer listContainer; private List<AVElement> avElements = new ArrayList<>(); private Surface surface; private VideoElementsListItemProvider videoElementsListItemProvider; @Override
public void onStart(Intent intent) {
super.onStart(intent);
super.setUIContent(ResourceTable.Layout_main_ability_slice);
addSurfaceProvider();
initPlayer();
initComponents();
} private void addSurfaceProvider() {
surfaceProvider = new SurfaceProvider(this); if (surfaceProvider.getSurfaceOps().isPresent()) {
surfaceProvider.getSurfaceOps().get().addCallback(new SurfaceCallBack());
surfaceProvider.pinToZTop(true);
} DirectionalLayout directionalLayout = (DirectionalLayout) findComponentById(ResourceTable.Id_directionalLayout);
directionalLayout.addComponent(surfaceProvider);
} private void initPlayer() {
videoPlayerPlugin = new VideoPlayerPlugin(getApplicationContext());
} private void initComponents() {
titleText = (Text) findComponentById(ResourceTable.Id_videoTitle_text);
Component refreshButton = findComponentById(ResourceTable.Id_refresh_list_button);
Component playButton = findComponentById(ResourceTable.Id_play_button);
Component pauseButton = findComponentById(ResourceTable.Id_pause_button);
Component seekButton = findComponentById(ResourceTable.Id_seek_button);
listContainer = (ListContainer) findComponentById(ResourceTable.Id_listContainer); playButton.setClickedListener((Component component) -> videoPlayerPlugin.startPlay());
pauseButton.setClickedListener(component -> videoPlayerPlugin.pausePlay());
seekButton.setClickedListener(component -> videoPlayerPlugin.seek());
refreshButton.setClickedListener(component -> refreshList());
listContainer.setItemClickedListener((listRoom, component, position, l) -> play(position));
videoElementsListItemProvider = new VideoElementsListItemProvider();
} private void play(int position) {
AVElement item = avElements.get(position);
String itemText = item.getAVDescription().getTitle().toString();
titleText.setText(itemText);
videoPlayerPlugin.startPlay(videoElementsListItemProvider.getItem(position), surface);
} private void refreshList() {
VideoElementManager videoElementManager = new VideoElementManager(getApplicationContext());
avElements = videoElementManager.getAvElements();
listContainer.setItemProvider(videoElementsListItemProvider);
} @Override
public void onStop() {
videoPlayerPlugin.release();
super.onStop();
} private void videoMetaData(Context context){
DataAbilityHelper helper = DataAbilityHelper.creator(context);
try {
DataAbilityPredicates predicates = new DataAbilityPredicates();
predicates.equalTo(AVStorage.Video.Media.DATA, "xxxxx");
ResultSet result = helper.query(AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI, new String[]{AVStorage.Video.Media.ID}, predicates);
if (result == null) {
return;
}
while (result.goToNextRow()) {
result.getInt(result.getColumnIndexForName(AVStorage.Video.Media.ID)); // Obtain the value of the id field.
}
result.close();
} catch (DataAbilityRemoteException e) {
}
} private void getMediaItemById(Context context, int id){
Uri uri = Uri.appendEncodedPathToUri(AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI, String.valueOf(id)); // The value of parameter id is the ID obtained in Step 1.
Player player = new Player(context);
DataAbilityHelper helper = DataAbilityHelper.creator(context);
// player.setSource(new Source(helper.openFile(uri, "r")));
} class SurfaceCallBack implements SurfaceOps.Callback {
@Override
public void surfaceCreated(SurfaceOps callbackSurfaceOps) {
if (surfaceProvider.getSurfaceOps().isPresent()) {
surface = surfaceProvider.getSurfaceOps().get().getSurface();
LogUtil.info(TAG, "surface set");
}
} @Override
public void surfaceChanged(SurfaceOps callbackSurfaceOps, int format, int width, int height) {
LogUtil.info(TAG, "surface changed");
} @Override
public void surfaceDestroyed(SurfaceOps callbackSurfaceOps) {
LogUtil.info(TAG, "surface destroyed");
}
} class VideoElementsListItemProvider extends BaseItemProvider {
@Override
public int getCount() {
return avElements.size();
} @Override
public AVElement getItem(int position) {
return avElements.get(position);
} @Override
public long getItemId(int position) {
return position;
} @Override
public Component getComponent(int position, Component component, ComponentContainer componentContainer) {
AVElement item = avElements.get(position);
if (component == null) {
String itemText = item.getAVDescription().getTitle().toString();
Text text = (Text) LayoutScatter.getInstance(MainAbilitySlice.this)
.parse(ResourceTable.Layout_list_item, null, false);
text.setText(itemText);
return text;
} else {
return component;
}
}
}
}
MainAbility.java:
import ohos.samples.videoplayer.slice.MainAbilitySlice;
import ohos.aafwk.ability.Ability;
import ohos.aafwk.content.Intent;
import ohos.bundle.IBundleManager;
import ohos.security.SystemPermission;public class MainAbility extends Ability {
private static final int REQUEST_CODE = 0X01; @Override
public void onStart(Intent intent) {
super.onStart(intent);
requestPermission();
super.setMainRoute(MainAbilitySlice.class.getName());
} private void requestPermission() {
if (verifyCallingOrSelfPermission(SystemPermission.READ_USER_STORAGE) != IBundleManager.PERMISSION_GRANTED) {
requestPermissionsFromUser(new String[]{SystemPermission.READ_USER_STORAGE}, REQUEST_CODE);
}
}
}
ability_main.xml:
<?xml version="1.0" encoding="utf-8"?>
<DirectionalLayout xmlns:ohos="http://schemas.huawei.com/res/ohos"
ohos:id="$+id:direction_layout"
ohos:width="match_parent"
ohos:height="match_parent"
ohos:orientation="vertical"> <Text
ohos:id="$+id:videoTitle_text"
ohos:text_size="15fp"
ohos:width="match_content"
ohos:text="$string:video"
ohos:layout_alignment="horizontal_center"
ohos:left_margin="5vp"
ohos:top_margin="20vp"
ohos:height="match_content"/> <DirectionalLayout
ohos:id="$+id:directionalLayout"
ohos:height="250vp"
ohos:alignment="center"
ohos:background_element="#000000"
ohos:width="match_parent"/> <DirectionalLayout
ohos:width="match_parent"
ohos:height="match_content"
ohos:top_margin="20vp"
ohos:left_margin="20vp"
ohos:right_margin="20vp"
ohos:layout_alignment="horizontal_center"
ohos:orientation="horizontal"> <Button
ohos:id="$+id:play_button"
ohos:text="$string:play"
ohos:width="0vp"
ohos:weight="1"
ohos:height="match_content"
ohos:padding="8vp"
ohos:right_margin="10vp"
ohos:background_element="$graphic:button_background"
ohos:text_size="20fp"/> <Button
ohos:id="$+id:pause_button"
ohos:text="$string:pause"
ohos:width="0vp"
ohos:weight="1"
ohos:height="match_content"
ohos:padding="8vp"
ohos:left_margin="10vp"
ohos:layout_alignment="vertical_center"
ohos:background_element="$graphic:button_background"
ohos:text_size="20fp"/>
</DirectionalLayout> <DirectionalLayout
ohos:width="match_parent"
ohos:height="match_content"
ohos:top_margin="10vp"
ohos:right_margin="20vp"
ohos:left_margin="20vp"
ohos:layout_alignment="horizontal_center"
ohos:orientation="horizontal"> <Button
ohos:id="$+id:seek_button"
ohos:text="$string:seek"
ohos:width="0vp"
ohos:weight="1"
ohos:padding="8vp"
ohos:height="match_content"
ohos:right_margin="10vp"
ohos:background_element="$graphic:button_background"
ohos:text_size="20fp"/> <Button
ohos:id="$+id:refresh_list_button"
ohos:text="$string:refresh_list"
ohos:width="0vp"
ohos:weight="1"
ohos:height="match_content"
ohos:padding="8vp"
ohos:left_margin="10vp"
ohos:background_element="$graphic:button_background"
ohos:text_size="20fp"/>
</DirectionalLayout> <ListContainer
ohos:id="$+id:listContainer"
ohos:top_padding="15vp"
ohos:height="0vp"
ohos:weight="1"
ohos:layout_alignment="horizontal_center"
ohos:width="match_parent"/>
</DirectionalLayout>
App Build Result
Tips and Tricks
- To store media files to the media database, use APIs in this section to transfer newly created or downloaded media files to the media scanning service. The scanning service reads metadata from the media files and adds the files to the database.
- The media scanning service can be invoked dynamically or statically.
- You can use the thumbnail tool for your application to obtain video and image thumbnails.
Conclusion
In this article, we have learned how to implement Media Data Management in Harmony OS application. In this application, I have explained that how user can get metadata of video files.
Thanks for reading this article. Be sure to like and comments to this article, if you found it helpful. It means a lot to me.
References
Harmony OS Doc:
