Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job

As a data analyst, my job is to get data from A to B, cleaned and ready for use. A common workflow for my team involves users uploading Excel files to a OneDrive folder. A Power Automate flow then syncs these files daily to a container in our Azure Storage Account . From there, my responsibility begins: Read the new Excel file from Blob Storage using Python. Process the data (clean, transform, apply business logic). Write the final data to an Azure SQL Database. I needed this to run on two trigg