Get the srcHeight and srcWidth of an image

App Building
, ,
1

Hey folks,

Really simple one - I'd like to get the srcHeight and srcWidth of an image within a script. I've loaded the image I need to get these values from in an image tag, but these properties are null. How can I get them to be set? Do I need to load the image data another way to find these values?

I'm building a tagger dashboard and need to map tag bounding boxes using an external format that is height and width-dependent (AWS Recognitions bounding box geometry)

2

I've figured out what I need, and documented my findings, and thoughts, for retools support team. This is what I've sent them:

My precise use case is to build a tagging dashboard for an ML image labeller, for QA and training purposes. I'm using AWS Rekognition for this.

My dashboard works by selecting an image to tag and displaying it for tagging in a bounding box component. The image is loaded from S3 using a signed URL.

I already have some labels to represent in this component when the image is loaded, so I need to map from the Rekognition geometry format to a geometry format supported by the Retool Bounding Box. I also want to save the new user-tagged labels in a way I understand within my system, so when a user presses a save button, I need to map the retool Bounding Box geometry format to the Rekognition geometry format.

This mapping is relatively straightforward - AWS Rekognition labels and geometry are represented with the following format:

{
  "CustomLabels": [
    {
      "Name": "Label Name",
      "Confidence": 98.13999938964844,
      "Geometry": {
        "BoundingBox": {
          "Width": 0.0888499990105629,
          "Height": 0.06835000216960907,
          "Left": 0.8645899891853333,
          "Top": 0.44211000204086304
        }
      }
    }
  ]
}

which needs to map to geometry format retool understands, like:

[
  {
    "label": "Label Name",
    "x": 2614.5201272964478,
    "y": 1782.5875282287598,
    "width": 268.6823970079422,
    "height": 275.58720874786377
  }
]

(this is where the awkward bit comes in)

To map between the two formats, I need to know the source image's height and width. in this case, the image I'm working with has a height of 4032 and a width of 3024. So to map to the format that retool understands, I do some maths like the below on the rekognition object example, giving me the retool formatted data above:

const retoolBoundingBox = {
  label: originObject.Name,
  x: ((originObject.Geometry.BoundingBox.Left/1) * data.width),
  y: ((originObject.Geometry.BoundingBox.Top/1) * data.height),
  width: ((originObject.Geometry.BoundingBox.Width/1) * data.width),
  height: ((originObject.Geometry.BoundingBox.Height/1) * data.height),
}

To access the height and width of the image, I would ideally like to use some metadata properties on the loaded bounding box component - however, there is no image metadata on this component, which would be incredibly useful. From a certain perspective property-wise, this might be a subclass of the image component. A feature to consider.

So, as these image properties (height & width) aren't available on the bounding box component, I tried loading the image from s3 using an image component. On this component, there are optional properties srcHeight and srcWidth - I wanted to access these to know the source images' true height and width.

However, I can't access these properties on an Image. So as a workaround, I'm creating and loading an Image using the browser API in a JavaScript Query, like below:

const getImageData = async () => {
  const imageSource = s3ImageSrc.data
  const img = await new Promise((resolve, reject) => {
    const i = new Image();
    i.onload = () => resolve(I);
    i.onerror = (err) => reject(err);
    i.src = imageSource.signedUrl;
  })

  return {height: img.naturalHeight, width: img.naturalWidth}
}

return Promise.resolve(getImageData())

This is not ideal, as it now requires me to load an image (potentially many megabytes) multiple times.

In an ideal world, the Bounding Boxes component that loads the image has the same properties as the Image component for the loaded image, and it provides me with a way to access the image's actual height and width.

3