After 30 days using Retool = {{ AI, ML, DALL-E, GPT-3, Azure Custom Vision & Wordpress }}

wpraiz_staff · November 10, 2022, 4:17am

Hello, beloveds,

This is my first post on the Retool forum, I hope to contribute more and more (as much as possible =D).

My name is José Ícaro, I graduated in Dentistry, but calm, I use a computer since 1994 and have studied Computer Science and other 3 universities, but only finished Dentistry (I do not take this as a merit, in fact, what matters most to me is to be happy and try to generate good things while we can).

I'm on Retool for +- 1 month and if I could summarize in a sentence what I am/we are going through (I am the founder of a Brazilian startup) I would say something like: "We found much more than we were looking for months ago to launch our MVP".

30 days of Retool

aiGrow (our MVP)

About the Project

A package of solutions for agencies and companies, B2B model of digital marketing where we intend to market the Apps/Panels in a customized way for companies nationwide and in Latin America and also offer customized Apps (in a second moment, for now only the model with the items below).

We have a few apps in validation but I was able to create them by myself (yes, I needed to validate all of this by myself due to security, time, staff, trust, and reliability of the data and because we have a small budget).

With Retool, in less than 7 days I had the scenario below validated:

Creating long texts (1,000+ words) with quality using GPT-3
Creation of customized texts (parts of content)
Sentiment analysis using:
-- Azure Language (Custom Language)
-- Azure OpenAI
-- Python Codes via Rest-API with an endpoint I created using FastAPI
Image creation with AI (DALL-E and maybe midjourney)
Automations and Integrations between WordPress (sites), Google Analytics and Google Search Console (we will leverage the initial scenario)

Core Integrations performed in aiGrow and Retool:
Retool + GPT-3 (OpenAI) + MongoDB
Retool + AirTable + MySQL
Retool + GPT-3 (OpenAI) + WordPress using Rest-API (wp-JSON)
Retool + WordPress Rest-API (auth using tokens)

Odontofy

About the Project

Solution with AI and ML, using datasets collected in partnership with two educational institutions (universities) and an autarchy of the Brazilian government called CFO (Federal Council of Dentistry) where the dentist send the image and the tool generates a result based on conditions displayed after requesting the Custom Vision API.
This project is on its second version, the first one used a simple form set up on WordPress and automation with Microsoft Power Automate and AI Builder from Microsoft 365.
Retool brought me the certainty that I would be able to do a much easier integration, using the Azure Custom Vision endpoint with the newly trained model.

Main Integrations performed in aiGrow and Retool:
Retool + Azure Custom Vision + Rest-API

Screenshots

aiGrow DALL-E (App Model)

About the Project

This is a sample app I decided to create to show on Youtube some of what we can do with Retool and also to pay homage to my 3-year-old son, João, because it's with him that I share my moments creating images with Dall-e since we entered Beta, in July.

Main Integrations made in aiGrow and Retool:
Retool + DALL-E API (OpenAI) + Rest-API + MongoDB

Thanks

alina.retool · November 10, 2022, 6:40pm

Hi @wpraiz_staff welcome to the community and thanks for sharing what you've built on Retool! This is a very interesting use case, and you're probably our first dentist-turned-developer on the platform. Very cool to see how dentists can use the app-- my followup question would be that once a dentist sends an image, what kinds of results are they expecting to receive? More specifically, what kinds of data is the Custom Vision API supplying?

Thanks again for sharing this and I look forward to seeing more of how Retool is helpful for you (and of course, we love feedback and questions too :))

wpraiz_staff · November 11, 2022, 2:32am

Hello, Alina!

I am very happy to receive my first reply on the Retool Forum and especially for asking about Odontofy!

Below are the main items about this project, a short video comparing the June model with the current one (lobe vs custom vision), in a non-objective way because the patterns were different and we classified the current dataset differently.

About the Project

This project is part of an initiative with a great friend who was my teacher and we became friends and companions of journey on Earth, Prof. Dr. Gustavo Emiliano, he is Forensic Doctor and Legal Dentistry (in Brazil Dentists are not doctors, but Prof Gustavo finished the faculty of Dentistry in Brazil, for example, and did his master's and doctorate in Portugal, that considers dentistry as part of medicine).

Prof. Gustavo works in Legal Dentistry, has some works with identification of victims of accidents, but the fact is that in 2019, the Federal Council of Dentistry of Brazil (CFO) released the use (posts / advertisements) with photos of the type before and after.

obs. Out: I just got an idea while creating the last paragraph, maybe we can create apps in Retool for clinics, I'll save it! Legal!

Briefly, after the release (regulatory and Federal LAW of the CFO) we had a significant increase in cases related to iatrogenic, ideological falsehood, fake News (people posing as dentists), use of generic internet photos, use of before and after clinics in another country (we had cases of dentists who were using images from social networks of other dentists in Canada).

In my view, health should not be commercialised, but shared, but unfortunately there are complex scenarios that run away from natural reason, and we have no choice.

When a patient suffers some biological loss or damage (reparable or irreparable) it is already too late, that is, who carried out the misleading advertisement, either with before and after or with other types of images, has already extracted the money he wanted. So we have a scenario associated with a legal consultancy that initially involves denunciation, verification, process and punishment.

In Brazil we suffer from corruption and in a way with habits inherited from colonization and other factors, so Brazilian justice tends to favor minorities.

I'm following cases where elderly patients, for example, have suffered mutilations and lost speech and face movements after seeing a Botox ad for facial harmonization where the professional did not even have a degree, let alone specialization.

Complaints, whether at regional level, by regional councils or by federal councils, are made manually.

I don't have this information about the actual number of cases that have increased, but it's something exponential.

Knowing this, let's get down to business:

About Dataset

A) About Dataset

Prof. Gustavo has a project in a State University (UERN) where he and two students, performed image collection, manually (yes, believe me), for almost 2 years, analyzing almost weekly Instagram posts on social networks of dentists who were selected by them through a criterion.

From March, with my entry, obviously I suggested dynamic scraping through a python setup, analyzing not only posts on Instagram, but also images on Facebook ads, google ads and TikTok, this via API.

So this would be the complete Odontofy project, we would have an initial dataset, filtered and validated for training and revalidation and availability.

In mid-June, I chose to create a simple prototype for Gustavo to perform a demonstration for a government representative. It was a setup created in 2-3 hours, but of course, in that scenario I was with a filtered dataset.

I had a redoubled work in both phases, because none of Gustavo's assistants, let alone him, understood what a dataset was actually and what the importance of having a significant amount of images.

The only options I could create something quickly and conveniently and that were already part of my day a few months ago, would be using an automation setup between WordPress, Power Automate and AI Builder.

For this dataset, I used Lobe in scenarios that were divided into about 4 or 5 training steps and that on the last day I had to correct the classification because, as the presentation would be for representatives of the CFO, and the government, we could not suggest that the images sent, even being a web prototype with a prediction "in validation", would in fact be wrong or right based on the law, but rather suggestive.

In this second part, using Azure Custom Vision, which I had never used until then, was my definitive choice after we entered Retool.

I confess that it is very difficult to be patient with people who have not yet undergoing sufficient technological maturation. For example, in this last dataset, there were 1,200 images received in a segmented, randomized way (by email, WhatsApp, through PowerPoint, google drive, etc.) and this delayed me a lot.

After being filtered (I performed a simplified analysis of duplicate images with Azure Custom Vision (self date) itself and then filtered the images by file size to see the duplicate items again.

Classification

B) About 230 images remained and we used 4 classifications, which were:

1 – Selfie (POSITIVE)

2 - Diagnosis/Completion of Treatment (POSITIVE)

3 - Course or Procedure (NEGATIVE)

4 - Identification of Equipment, Instruments, Materials or Biological Tissues present (NEGATIVE)

In the above scenario, I had to complement items 3 and 4 (Screenshot by Lightshot) because the model was completely balanced away from an acceptable "initial scenario, because we had about 140 selfie images, 50 Diagnosticimages and the rest of items 3 and 4.

I chose to do a manual scraping via the same browser, using similar patterns, filtering and arranging item by item in its respective classification.

Board of the Miro that I use since June for this project : https://prnt.sc/kiBrnhikDaMn

josefromeo123 · December 27, 2023, 9:41am

excellent never think about a dentist or any person related to medical field will do something like this