I was able to accomplish this by using the default Tailwind installation docs seen here: Installation - Tailwind CSS.
In my index.tsx file I imported ./output.css.
My file structure looks like so after following the Tailwind installation
Hope that helps