S3 Uploader not uploading files to bucket

Queries and Resources
1

Hi everyone,

I'm using the retool template for the S3 file uploader. I created a new retool S3 resource and replaced the demo 3 resource with my new S3 resource.

Listing and downloading files in the bucket works fine, but the file upload doesn't; when I check the network response it simply says '"Access Denied"'.

Locally, I wrote a small python script to try the data upload and that way and it actually works. Why wouldn't it work through the retool dashboard?? I also made sure to set the cors according to this retool community post:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE"
        ],
        "AllowedOrigins": [
            "https://*.retool.com"
        ],
        "ExposeHeaders": []
    },
    {
        "AllowedHeaders": [],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

Please help!
Many thanks,
Fina

2

Try adding the other methods
"AllowedMethods": ["PUT", "POST", "DELETE"] instead

3

@ScottR I added that in both sections (does that make sense though? Now everybody has put, post and delete rights) so my updated cors config looks like this:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE"
        ],
        "AllowedOrigins": [
            "https://*.retool.com"
        ],
        "ExposeHeaders": []
    },
    {
        "AllowedHeaders": [],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE",
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

but it still get an error response when I check the network tab:

{"error":true,"message":"Access Denied"}

What could be the issue??

4

@FinaSS This is likely not CORS related but due to the access credentials on the IAM role assigned to the user. Can you share what you have for those? An example in our docs in the dropdown here.

5

Hi Joe, thank you for getting back to me so quickly!
I'm able to upload files into the bucket using a small script I wrote in python&boto3&pandas. Since the script works I don't think it's an IAM permissions issue? I'm wondering if it has to do with the payload sent??

Any way, here's the policy that defines access to the S3 bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:GetObject",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": [
                "arn:aws:s3:::NAMEOFBUCKET/*",
                "arn:aws:s3:::NAMEOFBUCKET",
                "arn:aws:kms:MOREPRIVATEIDS/*",
                "arn:aws:kms:MOREPRIVATEIDS"
            ]
        }
    ]
}

I've changed my bucket cors config to the one suggested in your link:

[
  {
    "AllowedOrigins": ["https://*.retool.com"],
    "AllowedMethods": ["PUT", "POST", "DELETE"],
    "AllowedHeaders": ["*"]
  },
  {
    "AllowedOrigins": ["*"],
    "AllowedMethods": ["GET"]
  }
]

Still no success!

The payload of the retool dashboard looks like this:

acl
: 
"private"
bucketName
: 
""
embedded
: 
false
environment
: 
"production"
fileName
: 
"3bd2765b-7940-440f-9c8d-eadcd4db2a91.svg"
fileType
: 
"image/svg+xml"
pageName
: 
"PAGENAME"
resource
: 
null
resourceName
: 
"603a05ca-80ab-4245-be7b-49fd7b6abf3f"

Even when I explicitly set the bucketName to the name it returns an Access Denied error.

The script I wrote for uploading does this (this is the script that I mentioned above that works fine):

books_df.to_csv(
    f"s3://{AWS_S3_BUCKET}/{key}",
    index=False,
    storage_options={
        "key": AWS_ACCESS_KEY_ID,
        "secret": AWS_SECRET_ACCESS_KEY
    },
)

What could be the issue??

6

@joeBumbaca ahhh you were ABSOLUTELY RIGHT! It was the access permissions!
There was a little dropdown I hadn't seen at first in the link you sent called Create access credentials : Connect to Amazon S3 and S3-compatible services | Retool Docs
There are way more permissions in that config than in mine. I updated my s3 permissions to mirror those and that fixed it!

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketAcl",
                "s3:GetBucketCORS",
                "s3:GetBucketLocation",
                "s3:GetBucketLogging",
                "s3:GetBucketNotification",
                "s3:GetBucketPolicy",
                "s3:GetBucketWebsite",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging",
                "s3:PutObjectVersionAcl",
                "s3:PutObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME",
                "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}

(Still confused about why I can upload using my little local script but that's not important)

Thanks so much Joe!

1 Like
7

Excellent!! Glad that worked for you, should have sent you a screenshot of the dropdown I was referencing. But looks like you found it and are all set! Have a great day!

1 Like