I will show Java code that save results of select query on SQL Server into Azure Blob Storage.
- ▼1. Copy data on SQL Server into Azure Blob Storage
- ▼2. Prerequisites
- ▼3. Java code to copy results of a select query on SQL Server into Azure Blob Storage
- 3-1. Making a directory
- 3-2. Developing an Apache Maven project
- 3-3. Removing the existing App.java and AppTest.java
- 3-4. Starting Visual Studio Code
- 3-5. Adding SQL Server and Azure Blob Storage libraries into pom.xml (Ctrl+S)
- 3-6. Creating copysqlsrvtoblob.java code
- 3-7. Saving a csv file into Azure Blob Storage
- ▼4. Reference
▼1. Copy data on SQL Server into Azure Blob Storage
Azure Data Factory (ADF) is a good and easy way to copy data on SQL Server into Azure Blob Storage. please refer to this document. in this blog, I will show Java code to do it instead of ADF.
▼2. Prerequisites
We execute select query on SQL Server for Ubuntu.
2-1. Installation of SQL Server on Ubuntu
Ref: Quickstart: Install SQL Server and create a database on Ubuntu
2-2. Installation of Visual Studio Code on Ubuntu
sudo snap install --classic code
Ref: https://code.visualstudio.com/docs/setup/linux
2-3. Installation of Maven on Ubuntu
Ref: Apache Kafka Word Count – Java No.44
2-4. Creating a sample data on SQL Server
Ref: Python – Creating sample data in SQL Server on Ubuntu No.62
2-5. Executing a select query on SQL Server for Ubuntu
Ref: Java – Execute a Select query in SQL Server on Ubuntu No.8
▼3. Java code to copy results of a select query on SQL Server into Azure Blob Storage
3-1. Making a directory
Make a directory to develop Java application
mkidr sqlsrvtest
cd sqlsrvtest
3-2. Developing an Apache Maven project
cd sqlsrvtest
mvn archetype:generate -DinteractiveMode=false -DgroupId=org.example.sqlsrv -DartifactId=sqlSrvOutput -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4
(A part of output) xxxx [INFO] Generating project in Batch mode [INFO] Archetype repository not defined. Using the one from [org.apache.maven.archetypes:maven-archetype-quickstart:1.4] found in catalog remote [INFO] ------------------------------------------------------------------- [INFO] Using following parameters for creating project from Archetype: maven-archetype-quickstart:1.4 [INFO] -------------------------------------------------------------------- [INFO] Parameter: groupId, Value: org.example.sqlsrv [INFO] Parameter: artifactId, Value: sqlSrvOutput [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Parameter: package, Value: org.example.sqlsrv [INFO] Parameter: packageInPathFormat, Value: org/example/sqlsrv [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Parameter: package, Value: org.example.sqlsrv [INFO] Parameter: groupId, Value: org.example.sqlsrv [INFO] Parameter: artifactId, Value: sqlSrvOutput [INFO] Project created from Archetype in dir: /home/xxx/sqlSrvOutput [INFO] -------------------------------------------------------------------- [INFO] BUILD SUCCESS [INFO] -------------------------------------------------------------------- [INFO] Total time: 3.858 s [INFO] Finished at: 2022-11-12T22:35:18+09:00 [INFO] --------------------------------------------------------------------
3-3. Removing the existing App.java and AppTest.java
rm ./sqlSrvOutput/src/main/java/org/example/sqlsrv/App.java
rm ./sqlSrvOutput/src/test/java/org/example/sqlsrv/AppTest.java
3-4. Starting Visual Studio Code
$code .
3-5. Adding SQL Server and Azure Blob Storage libraries into pom.xml (Ctrl+S)
<properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> <dependency> <groupId>com.microsoft.sqlserver</groupId> <artifactId>mssql-jdbc</artifactId> <version>11.2.1.jre8</version> </dependency> <dependency> <groupId>com.azure</groupId> <artifactId>azure-storage-blob</artifactId> <version>12.13.0</version> </dependency> </dependencies>
3-6. Creating copysqlsrvtoblob.java code
package org.example.sqlsrv;
import java.io.FileNotFoundException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import com.azure.storage.blob.*;
public class copysqlsrvtoblob {
// select data from SQL Server
public static void main(String[] args) throws SQLException {
String connectStr = "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=yourkey;EndpointSuffix=core.windows.net";
String containerName = "files";
String fileName = "sqlSrvOutput.csv";
String localPath = "/home/xxx/sqlsrvtoblob/sqlSrvOutput/";
// Get a reference to a container
BlobServiceClient BlobServiceClient = new BlobServiceClientBuilder().connectionString(connectStr).buildClient();
BlobContainerClient containerClinet = BlobServiceClient.getBlobContainerClient(containerName);
String url = "jdbc:sqlserver://localhost:1433;databaseName=sampledb;user=sa;password=yourpassowrd;encrypt=false";
Connection conn = DriverManager.getConnection(url);
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("select * from dbo.sampletable");
List<String> list = new ArrayList<String>();
while (rs.next()) {
String ID = rs.getString("id");
String NUM = rs.getString("num");
String cdatetime = rs.getString("cdatetime");
String note = rs.getString("note");
String str = ID + "," + NUM + "," + cdatetime + "," + note;
list.add(str);
}
rs.close();
stmt.close();
conn.close();
// save local file
String filename = "/home/xxx/sqlsrvtoblob/sqlSrvOutput/sqlSrvOutput.csv";
try {
java.io.PrintWriter output = new java.io.PrintWriter(filename);
for (int i = 0; i < list.size(); i++) {
output.println(list.get(i));
}
output.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
// Upload a file to the blob
BlobClient blobClient = containerClinet.getBlobClient(fileName);
System.out.println("Uploading to Azure Blob storage as blob:\n\t" + blobClient.getBlobUrl());
blobClient.uploadFromFile(localPath + "/" + fileName);
}
}
3-7. Saving a csv file into Azure Blob Storage
(A part of output) ---------------------------------------- SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Uploading to Azure Blob storage as blob: https://xxx.blob.core.windows.net/files/sqlSrvOutput.csv ----------------------------------------
sqlSrvOutput.csv contains below.
1,0,2022-10-20 16:13:13.477,init 2,1,2022-10-20 16:12:11.777,a 3,2,2022-10-20 16:12:11.777,ah 4,3,2022-10-20 16:12:11.777,lam 5,4,2022-10-20 16:12:11.777,nddv 6,5,2022-10-20 16:12:11.777,vdhpw 7,6,2022-10-20 16:12:11.777,fdfszy 8,7,2022-10-20 16:12:11.777,rgofcgt 9,8,2022-10-20 16:12:11.777,hezwdqae 10,9,2022-10-20 16:12:11.777,bywqcunva
▼4. Reference
1. Java – Execute a Select query in SQL Server on Ubuntu No.8
2. Quickstart: Install SQL Server and create a database on Ubuntu
3. VS Code installation https://code.visualstudio.com/docs/setup/linux
4. Maven installation Apache Kafka Word Count – Java No.44
5. Python – Creating sample data in SQL Server on Ubuntu No.62
6. Java Azure Blob Storage へファイルのアップロード No.69
That’s all. Have a nice day ahead !!!