Displaying an image from USB on a display using Harmony.

This forum handles questions and discussions concerning MPLAB Harmony integrated firmware framework and all associated Libraries and components.

Displaying an image from USB on a display using Harmony.

Postby KTrenholm » Thu Dec 05, 2019 7:52 pm

Hey all,

First things first, here's what I'm using:
MPLAB X v5.30
XC32 v2.30
Harmony 3
PIC32MZDA Starter Kit (Non-Crypto, Internal DDR)
MEBII Display Kit

I have a project in which I need to read a USB Mass Storage Device for images, and show these images on a display. I've only been using harmony for a couple weeks, and it's been quite the trip.

I've had some success so far manually reading a 24-bit bitmap, converting it over to 32-bit ARGB, and manually drawing the bitmap using the n2d_blit function (as chronicled here: https://www.microchip.com/forums/m1120120.aspx).

Where I'm running into some issues is making the transition between images clean. I have a weird visual artifact that looks like the image is being drawn wrapped around the display (up to about 1/3 of the image), and then "slides" into the correct place. It almost seems like a timing issue of some kind. This isn't a showstopper (this project is for a display test rig, so clean transitions between images isn't a huge problem, it actually almost looks like a feature!) but I'd like to figure out why it's behaving like this, and find a workaround if possible.

I initially thought that maybe performing the read from USB and displaying the image all in one go might be messing with the timing for drawing the image, so I split up the read and the blit into two separate states (and also checked to make sure no draw was already taking place) so the image is read and displayed on successive calls of APP_Tasks(). This unfortunately didn't do anything to solve the issue.

It was suggested in another thread (https://www.microchip.com/forums/m1120944.aspx#1121189) to maybe try using the harmony Image Widget rather than a direct blit. This seems promising but I can't seem to get it to work correctly, and the tips I'm getting just seem to cycle back to "let harmony configurator pre-process the images to a bin file before putting them on the USB MSD", which is not an option. Requirement is to have the images be loaded as images onto a USB MSD. We can't really tell the customer that they need to preprocess images with harmony configurator. I should definitely be able to at least take a bitmap and put it into the image widget at runtime.

From what I understand, to process an image for use with the image widget, I need to use laUtils_PreprocessImage() to process the image data into a format where it can be written to the widget.

What I want to do is:
- Load image from USB MSD to DDR
- Process this image to a "staging" area elsewhere in DDR memory
- Write the newly processed image to the Image Widget

My code to attempt this:
Code: Select all
#define IMAGE_DATA_ADDR 0xA827E800  //Processed image is placed here by laUtils_PreprocessImage()
#define IMAGE_TEMP_ADDR 0xA817E000  //Image is read from USB to this address in DDR

uint32_t * ptrImageData = (uint32_t *)IMAGE_DATA_ADDR;
uint32_t * ptrImageTemp = (uint32_t *)IMAGE_TEMP_ADDR;

GFXU_ImageAsset ImageAsset_Pre;
GFXU_ImageAsset ImageAsset_Post;

/*BITMAP_DATA contains the file handle, size of the file, and the index of the beginning of pixel data (currently unused)*/
void SetupBitmapData (BITMAP_DATA * bmp_info){
    uint32_t size = SYS_FS_FileSize(bmp_info->fs_handle);
   
    SYS_FS_FileRead(bmp_info->fs_handle, ptrImageTemp, size);
   
    ImageAsset_Pre.header.type = GFXU_ASSET_TYPE_IMAGE;
    ImageAsset_Pre.header.dataLocation = GFXU_ASSET_LOCATION_ID_INTERNAL;
    ImageAsset_Pre.header.dataAddress = (void *)ptrImageTemp;
    ImageAsset_Pre.header.dataSize = size;
   
    ImageAsset_Pre.format = GFXU_IMAGE_FORMAT_RAW;          //bitmap - raw pixel data
    ImageAsset_Pre.width = IMAGE_WIDTH;
    ImageAsset_Pre.height = IMAGE_HEIGHT;
    ImageAsset_Pre.bufferWidth = IMAGE_WIDTH;
    ImageAsset_Pre.bufferHeight = IMAGE_HEIGHT;
    ImageAsset_Pre.colorMode = GFX_COLOR_MODE_RGB_888;      //24-bit bitmap
    ImageAsset_Pre.compType = GFXU_IMAGE_COMPRESSION_NONE;
    ImageAsset_Pre.flags = 0;
    ImageAsset_Pre.mask = 0x0;
    ImageAsset_Pre.palette = GFX_NULL;
   
    /*Process image to IMG_DATA_ADDR in DDR as ARGB8888*/
    laUtils_PreprocessImage(&ImageAsset_Pre, IMAGE_DATA_ADDR, GFX_COLOR_MODE_ARGB_8888, GFX_FALSE, NULL);
   
    /*Setup ImageAsset_Post to describe our new post-processed image data*/
    ImageAsset_Post.header.type = GFXU_ASSET_TYPE_IMAGE;
    ImageAsset_Post.header.dataLocation = GFXU_ASSET_LOCATION_ID_INTERNAL;
    ImageAsset_Post.header.dataAddress = (void *)ptrImageData;
    ImageAsset_Post.header.dataSize = size + (IMAGE_HEIGHT * IMAGE_WIDTH);  //+1 alpha byte for each pixel of the image
   
    ImageAsset_Post.format = GFXU_IMAGE_FORMAT_RAW;          //bitmap - raw pixel data
    ImageAsset_Post.width = IMAGE_WIDTH;
    ImageAsset_Post.height = IMAGE_HEIGHT;
    ImageAsset_Post.bufferWidth = IMAGE_WIDTH;
    ImageAsset_Post.bufferHeight = IMAGE_HEIGHT;
    ImageAsset_Post.colorMode = GFX_COLOR_MODE_ARGB_8888;      //Process should have made image data ARGB8888
    ImageAsset_Post.compType = GFXU_IMAGE_COMPRESSION_NONE;
    ImageAsset_Post.flags = 0;
    ImageAsset_Post.mask = 0x0;
    ImageAsset_Post.palette = GFX_NULL;
}


The image is then displayed on the next call of APP_Tasks():
Code: Select all
laImageWidget_SetImage((laImageWidget*)img_backImage, &ImageAsset_Post);


All this gives me is a white screen for all my images. Would anyone be able to tell me if I'm on the right track here? Am I missing something?
User avatar
KTrenholm
 
Posts: 26
Joined: Mon Sep 10, 2018 3:57 pm
PIC experience: Professional 5+ years with MCHP products

Re: Displaying an image from USB on a display using Harmony.

Postby KTrenholm » Thu Dec 05, 2019 10:38 pm

After looking at the actual code for laUtils_PreprocessImage(), it looks like it actually takes your passed in GFXU_ImageAsset * and updates it with the new address and even updates it with the size of the result. Note the following:
Code: Select all
laResult laUtils_PreprocessImage(GFXU_ImageAsset* img,
                                uint32_t destAddress,
                                GFX_ColorMode destMode,
                                GFX_Bool padBuffer,
                                GFXU_MemoryIntf * memIntf){

/*snip*/
img->header.dataAddress = (void*)destAddress;
    img->header.dataLocation = 0;
    img->header.dataSize = buf.buffer_length;
    img->bufferWidth = buf.size.width;
    img->bufferHeight = buf.size.height;
    img->format = GFXU_IMAGE_FORMAT_RAW;
    img->compType = GFXU_IMAGE_COMPRESSION_NONE;
}


So it looks like I shouldn't actually need a second "post-process" GFXU_ImageAsset to write to the widget, it's updating the one I passed in for me.

As such, code is now:
Code: Select all
void SetupBitmapData (BITMAP_DATA * bmp_info){
    uint32_t size = SYS_FS_FileSize(bmp_info->fs_handle);
   
    SYS_FS_FileRead(bmp_info->fs_handle, ptrImageTemp, size);
   
    ImageAsset.header.type = GFXU_ASSET_TYPE_IMAGE;
    ImageAsset.header.dataLocation = GFXU_ASSET_LOCATION_ID_INTERNAL;
    ImageAsset.header.dataAddress = (void *)ptrImageTemp;
    ImageAsset.header.dataSize = size;
   
    ImageAsset.format = GFXU_IMAGE_FORMAT_RAW;          //bitmap - raw pixel data
    ImageAsset.width = IMAGE_WIDTH;
    ImageAsset.height = IMAGE_HEIGHT;
    ImageAsset.bufferWidth = IMAGE_WIDTH;
    ImageAsset.bufferHeight = IMAGE_HEIGHT;
    ImageAsset.colorMode = GFX_COLOR_MODE_RGB_888;      //24-bit bitmap
    ImageAsset.compType = GFXU_IMAGE_COMPRESSION_NONE;
    ImageAsset.flags = 0;
    ImageAsset.mask = 0x0;
    ImageAsset.palette = GFX_NULL;
   
    /*Process image to IMG_DATA_ADDR in DDR as ARGB8888*/
    laUtils_PreprocessImage(&ImageAsset, IMAGE_DATA_ADDR, GFX_COLOR_MODE_ARGB_8888, GFX_FALSE, NULL);
}


The fact that it modifies your passed in GFXU_ImageAsset is, of course, not mentioned in the documentation for the function:
preprocess.JPG
preprocess.JPG (140.92 KiB) Viewed 5182 times



This didn't actually fix the problem (still nothing but white in the imageWidget), but at least I'm a step closer to maybe doing this correctly.
User avatar
KTrenholm
 
Posts: 26
Joined: Mon Sep 10, 2018 3:57 pm
PIC experience: Professional 5+ years with MCHP products

Re: Displaying an image from USB on a display using Harmony.

Postby KTrenholm » Fri Dec 06, 2019 5:20 pm

Had a thought this morning that perhaps laUtils_PreprocessImage() was maybe expecting the image header information to be stripped, leaving only pixel data. Gave it a shot and no change in behavior:

Code: Select all
static void SetupBitmapData (BITMAP_DATA * bmp_info){
    uint32_t size = SYS_FS_FileSize(bmp_info->fs_handle) - bmp_info->px_index;
   
    SYS_FS_FileSeek(bmp_info->fs_handle, bmp_info->px_index, SYS_FS_SEEK_SET); /*Set file pointer to first pixel of image data after header*/
   
    SYS_FS_FileRead(bmp_info->fs_handle, ptrImageTemp, size-bmp_info->px_index);
   
    ImageAsset.header.type = GFXU_ASSET_TYPE_IMAGE;
    ImageAsset.header.dataLocation = GFXU_ASSET_LOCATION_ID_INTERNAL;
    ImageAsset.header.dataAddress = (void *)ptrImageTemp;
    ImageAsset.header.dataSize = size-bmp_info->px_index;
   
    ImageAsset.format = GFXU_IMAGE_FORMAT_RAW;          //bitmap - raw pixel data
    ImageAsset.width = IMAGE_WIDTH;
    ImageAsset.height = IMAGE_HEIGHT;
    ImageAsset.bufferWidth = IMAGE_WIDTH;
    ImageAsset.bufferHeight = IMAGE_HEIGHT;
    ImageAsset.colorMode = GFX_COLOR_MODE_RGB_888;      //24-bit bitmap
    ImageAsset.compType = GFXU_IMAGE_COMPRESSION_NONE;
    ImageAsset.flags = 0;
    ImageAsset.mask = 0x0;
    ImageAsset.palette = GFX_NULL;
   
    /*Process image to IMG_DATA_ADDR in DDR as ARGB8888*/
    laUtils_PreprocessImage(&ImageAsset, IMAGE_DATA_ADDR, GFX_COLOR_MODE_ARGB_8888, GFX_FALSE, NULL);
}


Another weird side effect of using laImageWidget_SetImage() seems to be that it prevents functions like n2d_fill() from working. Was not a problem and worked as expected when I was just blitting the image out.
User avatar
KTrenholm
 
Posts: 26
Joined: Mon Sep 10, 2018 3:57 pm
PIC experience: Professional 5+ years with MCHP products

Re: Displaying an image from USB on a display using Harmony.

Postby KTrenholm » Fri Dec 06, 2019 10:34 pm

I have uploaded my current version of the project (https://drive.google.com/file/d/1bJOt7KTi9ofSX7Z17sKvJtEKyrKV2I3s/view). Probably more insightful than explaining how everything works and potentially missing something critical. Maybe something will get noticed that I'm missing when all the code is shown. Do pardon the mess of the code in places, I've been doing quite a bit of tinkering with this for a bit trying to get it working. There's a #define USE_BLIT at the top of app.c that switches between using direct blit to layer0 and attempting to use the ImageWidget. There's another #define USE_BMP_HEADER that determines if the Bitmap header should be kept or stripped before being passed to Preprocess(). Files on the USB MSD are expected to be 8.3 filename, 24-bit bitmaps.

Bitmap gets read in from USB in the USB_STATE_OPEN_FILE state. Sent to the ImageWidget in the USB_STATE_DISPLAY_FILE state. The fade for the text widget on layer 1 is a little busted, and not working exactly as intended, but the label itself works to read out the filename of the image being put on the display.

As far as I can tell, this really should be as simple as constructing a GFXU_ImageAsset from the read in data to the laImageWidget_SetImage() function, but it just can't seem to figure out why it doesn't display. There has to be something going on that I'm not seeing that's throwing a wrench into the works.
User avatar
KTrenholm
 
Posts: 26
Joined: Mon Sep 10, 2018 3:57 pm
PIC experience: Professional 5+ years with MCHP products

Re: Displaying an image from USB on a display using Harmony.

Postby Drazan » Fri Dec 04, 2020 1:57 pm

Honestly, you made me speechless with your post... It was like I entered IT-courses, you know. So many unknown combinations of numbers. I guess you must be a very professional specialist in this field. Anyway, I am ashamed to tell you one thing... I still don't understand what is the brand you called 'Harmony', sorry for that. Is it an app you use for crafting images? As for me, I know several resources which allow you to upscale image online with ease. But I was truly shocked when I saw I can upscale any picture by up to 800%. It's just inexplicable, am I right?
Last edited by Drazan on Mon Dec 07, 2020 3:15 pm, edited 1 time in total.
Drazan
 
Posts: 1
Joined: Fri Dec 04, 2020 1:55 pm

Re: Displaying an image from USB on a display using Harmony.

Postby AussieSusan » Mon Dec 07, 2020 2:36 am

This thread is about 1 year old.
Harmony is part of the Microchip offerings - check out their web site.
Susan
AussieSusan
Verified identity
 
Posts: 173
Joined: Mon Jun 16, 2014 4:45 am
PIC experience: Experienced Hobbyist


Return to MPLAB Harmony

Who is online

Users browsing this forum: No registered users and 3 guests

cron