This is an exercise in scrapping Data off of amazon.com and exporting to a .csv. I had to attempt some things out of my comfort zone so I had to do some research. I posted the videos I sued for reference at the bottom of this page

To begin I to connect to import the libs and connect to the website

• After the data is stored in soup variable, the code is ready to produce an agent that will look for the data. I then open the browser and I look for the product’s title (In this case, a video game)

• After some attempts, the requests fuction gets blocked by responde code 503
• This pretty much invalidates the work so far as this method is blocked by Amazon’s bot policy and prevents the User-Agent to access the data properly
• As a refresher, this is the structure of the HTML (Source)

Looking for solutions, I’ve found a lib called Selenium which automates web browser work

• This is how to change the settings to make it work in the background:

• By using Selenium web driver I completely bypassed the need for a User-Agent by having a automated browser window.
• This might create issues with Captcha and/or 2FA but for the first attempt I can bypass those manually and try again.
• Ultimately I was able to implement a rudimentary timer so I can run this script daily if needed