Java – Save the results of select query on SQL Server into Azure Blob Storage No.75

I will show Java code that save results of select query on SQL Server into Azure Blob Storage.

▼1. Copy data on SQL Server into Azure Blob Storage

Azure Data Factory (ADF) is a good and easy way to copy data on SQL Server into Azure Blob Storage. please refer to this document. in this blog, I will show Java code to do it instead of ADF.

Ref: Copy and transform data to and from SQL Server by using Azure Data Factory or Azure Synapse Analytics


▼2. Prerequisites

We execute select query on SQL Server for Ubuntu.

2-1. Installation of SQL Server on Ubuntu

Ref: Quickstart: Install SQL Server and create a database on Ubuntu

2-2. Installation of Visual Studio Code on Ubuntu

sudo snap install --classic code

Ref: https://code.visualstudio.com/docs/setup/linux

2-3. Installation of Maven on Ubuntu

Ref: Apache Kafka Word Count – Java No.44

2-4. Creating a sample data on SQL Server

Ref: Python – Creating sample data in SQL Server on Ubuntu No.62

2-5. Executing a select query on SQL Server for Ubuntu

Ref: Java – Execute a Select query in SQL Server on Ubuntu No.8


▼3. Java code to copy results of a select query on SQL Server into Azure Blob Storage

3-1. Making a directory

Make a directory to develop Java application

mkidr sqlsrvtest
cd sqlsrvtest

3-2. Developing an Apache Maven project

cd sqlsrvtest
mvn archetype:generate -DinteractiveMode=false -DgroupId=org.example.sqlsrv -DartifactId=sqlSrvOutput -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4
(A part of output)
xxxx
[INFO] Generating project in Batch mode
[INFO] Archetype repository not defined. Using the one from [org.apache.maven.archetypes:maven-archetype-quickstart:1.4] found in catalog remote
[INFO] -------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: maven-archetype-quickstart:1.4
[INFO] --------------------------------------------------------------------
[INFO] Parameter: groupId, Value: org.example.sqlsrv
[INFO] Parameter: artifactId, Value: sqlSrvOutput
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: org.example.sqlsrv
[INFO] Parameter: packageInPathFormat, Value: org/example/sqlsrv
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: org.example.sqlsrv
[INFO] Parameter: groupId, Value: org.example.sqlsrv
[INFO] Parameter: artifactId, Value: sqlSrvOutput
[INFO] Project created from Archetype in dir: /home/xxx/sqlSrvOutput
[INFO] --------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------
[INFO] Total time:  3.858 s
[INFO] Finished at: 2022-11-12T22:35:18+09:00
[INFO] --------------------------------------------------------------------

3-3. Removing the existing App.java and AppTest.java

rm ./sqlSrvOutput/src/main/java/org/example/sqlsrv/App.java
rm ./sqlSrvOutput/src/test/java/org/example/sqlsrv/AppTest.java

3-4. Starting Visual Studio Code

$code .

3-5. Adding SQL Server and Azure Blob Storage libraries into pom.xml (Ctrl+S)

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.microsoft.sqlserver</groupId>
      <artifactId>mssql-jdbc</artifactId>
      <version>11.2.1.jre8</version>
    </dependency>
    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-storage-blob</artifactId>
      <version>12.13.0</version>
    </dependency>   
  </dependencies>

3-6. Creating copysqlsrvtoblob.java code

package org.example.sqlsrv;

import java.io.FileNotFoundException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import com.azure.storage.blob.*;

public class copysqlsrvtoblob {
    // select data from SQL Server
    public static void main(String[] args) throws SQLException {
        String connectStr = "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=yourkey;EndpointSuffix=core.windows.net";
        String containerName = "files";
        String fileName = "sqlSrvOutput.csv";
        String localPath = "/home/xxx/sqlsrvtoblob/sqlSrvOutput/";

        // Get a reference to a container
        BlobServiceClient BlobServiceClient = new BlobServiceClientBuilder().connectionString(connectStr).buildClient();
        BlobContainerClient containerClinet = BlobServiceClient.getBlobContainerClient(containerName);

        String url = "jdbc:sqlserver://localhost:1433;databaseName=sampledb;user=sa;password=yourpassowrd;encrypt=false";
        Connection conn = DriverManager.getConnection(url);
        Statement stmt = conn.createStatement();
        ResultSet rs = stmt.executeQuery("select * from dbo.sampletable");
        List<String> list = new ArrayList<String>();
        while (rs.next()) {
            String ID = rs.getString("id");
            String NUM = rs.getString("num");
            String cdatetime = rs.getString("cdatetime");
            String note = rs.getString("note");
            String str = ID + "," + NUM + "," + cdatetime + "," + note;
            list.add(str);
        }
        rs.close();
        stmt.close();
        conn.close();

        // save local file
        String filename = "/home/xxx/sqlsrvtoblob/sqlSrvOutput/sqlSrvOutput.csv";
        try {
            java.io.PrintWriter output = new java.io.PrintWriter(filename);
            for (int i = 0; i < list.size(); i++) {
                output.println(list.get(i));
            }
            output.close();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }

        // Upload a file to the blob
        BlobClient blobClient = containerClinet.getBlobClient(fileName);
        System.out.println("Uploading to Azure Blob storage as blob:\n\t" + blobClient.getBlobUrl());
        blobClient.uploadFromFile(localPath + "/" + fileName);
    }
}

3-7. Saving a csv file into Azure Blob Storage

(A part of output)
----------------------------------------
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Uploading to Azure Blob storage as blob:
	https://xxx.blob.core.windows.net/files/sqlSrvOutput.csv
----------------------------------------

sqlSrvOutput.csv contains below.

1,0,2022-10-20 16:13:13.477,init
2,1,2022-10-20 16:12:11.777,a
3,2,2022-10-20 16:12:11.777,ah
4,3,2022-10-20 16:12:11.777,lam
5,4,2022-10-20 16:12:11.777,nddv
6,5,2022-10-20 16:12:11.777,vdhpw
7,6,2022-10-20 16:12:11.777,fdfszy
8,7,2022-10-20 16:12:11.777,rgofcgt
9,8,2022-10-20 16:12:11.777,hezwdqae
10,9,2022-10-20 16:12:11.777,bywqcunva

▼4. Reference

1. Java – Execute a Select query in SQL Server on Ubuntu No.8
2. Quickstart: Install SQL Server and create a database on Ubuntu
3. VS Code installation https://code.visualstudio.com/docs/setup/linux
4. Maven installation Apache Kafka Word Count – Java No.44
5. Python – Creating sample data in SQL Server on Ubuntu No.62
6. Java Azure Blob Storage へファイルのアップロード No.69

That’s all. Have a nice day ahead !!!

Leave a Reply

Your email address will not be published. Required fields are marked *