How to put the results of select query for MySQL into Azure Blob Storage using Python is shown in this blog.
▼1. Saving data in MySQL into Azure Blob Storage
In the past, how to save data in MySQL into a local file using Python was shown in the following blog. in this case, the target of saving data is Azure Blob Storage.
Python – Save data of MySQL into a local file No.99
▼2. Prerequisites
How to create an environment to run python code for MySQL is shown in this blog.
Python – Connect to MySQL on Ubuntu 20.04 by pyodbc No.98
▼3. Python code to save data in MySQL into Azure Blob Storage
3-1. Creating python code
Python code “savemysqldata2blob.py” is created to save results of select statement for MySQL into Azure Blob Storage. according to your environment, parameters should be changed. in this code, “result5.txt” is saved as a local file and then “results10.txt” is saved into Azure Blob Storage.
import pyodbc
import random
import time, datetime, string
driver = '{MYSQL ODBC 8.0 Driver}'
server = 'localhost'
dbname = 'testdb'
user = 'hiveuser'
pw = 'hivepassword'
query = "select * from testtbl"
success = False
retryc = 0
max_retries = 3
connection_string = 'DefaultEndpointsProtocol=https;AccountName=xxxx;EndpointSuffix=core.windows.net'
local_filename = '/home/xxx/results5.txt'
container_name = "files"
blob_name = "results10.txt"
from azure.storage.blob import BlobServiceClient
blob_service_client = BlobServiceClient.from_connection_string(connection_string)
# create a container and a blob client
container_client = blob_service_client.get_container_client(container_name)
blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)
while not success and retryc < max_retries:
try:
conn = pyodbc.connect('DRIVER='+driver+';SERVER='+server+';PORT=3306;DATABASE='+dbname+';UID='+user+';PWD='+pw+';')
cursor = conn.cursor()
cursor.execute(query)
row = cursor.fetchone()
# save the results each line to a file
with open(local_filename, 'w') as f:
while row:
f.write(str(row[0]) + " " + str(row[1]) + "\n")
row = cursor.fetchone()
success = True
except pyodbc.Error as ex:
retryc += 1
print("Current Timestamps:" + str(datetime.datetime.now()) + "retry count: " + str(retryc))
print("==========================")
print("Error: " + str(ex))
print("==========================")
time.sleep(15)
# upload the file to blob storage
with open(local_filename, "rb") as data:
blob_client.upload_blob(data)
# list the blobs in the container
blob_list = container_client.list_blobs()
for blob in blob_list:
print("\t" + blob.name)
# if success is success, close the cursor and connection
if success:
cursor.close()
conn.close()
3-2. Results of this code
A file name that is saved into Azure Blob Storage is listed by executing this Python code.
python3 savemysqldata2blob.py
results10.txt
“results10.txt” contains below.
cat results10.txt
1 1
2 2
3 3
▼4. Reference
- Python – Save data of MySQL into a local file No.99
- Python – Connect to MySQL on Ubuntu 20.04 by pyodbc No.98
- Python in Visual Studio Code
That’s all. Have a nice day ahead !!!