Java JSON ファイルのデータを Azure Cosmos DB (SQL API) に入れる No.22

自己投資としてチャレンジしている内容を Blog で公開しています。

今回は JSON ファイルのデータを読み Cosmos DB (SQL API) に入れる方法について紹介します。

▼1. JSONファイルのデータを Cosmos DB (SQL API) に入れる方法

最初に JSON からデータを読むために FileReaderクラスを利用します。読んだデータは JSONParser で構文解析 (parse) して JSONArray に入れ、Cosmos DB へのデータ挿入用に作成した Family のクラスの引数として渡します。Family のクラスのインスタンス family を item として Cosmos DB のコレクション (コンテイナー) に入れています。今回の例では parsePersonObject のメソッドを作成し、その中で JSONArray から抽出した JSONObject の値を Cosmos DB にデータを挿入しています。


▼2. 事前準備

2-1. Cosmos DB トライアルアカウントの作成

以下のサイトから無料で利用できる Azure Cosmos DB を用意します。Azure を利用するために必要な Subscription を用意する必要もありません。
https://azure.microsoft.com/ja-jp/try/cosmosdb/

2-2. pom.xml にて json-simple の maven 依存関係を追記

<dependency>
    <groupId>com.googlecode.json-simple</groupId>
    <artifactId>json-simple</artifactId>
    <version>1.1.1</version>
</dependency>

2-3. Azure Cosmos DB Java SDK v4 4.8.0 を利用

pom.xml に以下のバージョンが指定されていることを確認します。

<dependency>
    <groupId>com.googlecode.json-simple</groupId>
    <artifactId>json-simple</artifactId>
    <version>1.1.1</version>
</dependency>

参考情報: Azure Cosmos DB Java SDK v4 for Core (SQL) API: release notes and resources
https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-java-v4

2-4. test.json の中身を確認

[{"person":{"isRegistered":true,"LastName":"Travase"},"id":"7"},{"person":{"isRegistered":true,"LastName":"Ken"},"id":"8"}]

参考情報: Java JSON ファイルの作成とデータの読み込み No.21


▼3. JSON ファイルのデータを Cosmos DB (SQL API) に書き込むコード

3-1. Java コードの作成

JSON ファイルのデータを読み Cosmos DB (SQL API) の collection (container) に挿入します。最後に挿入したデータ (item) を確認しています。CosmosDBAccountName、DatabaseName、end_point 、master_key、containerName はそれぞれ利用する値に書き換えます。

import com.azure.cosmos.*;
import com.azure.cosmos.models.CosmosItemRequestOptions;
import com.azure.cosmos.models.CosmosItemResponse;
import com.azure.cosmos.models.CosmosQueryRequestOptions;
import com.azure.cosmos.models.PartitionKey;
import com.azure.cosmos.util.CosmosPagedIterable;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.stream.Collectors;

public class ReadJsonInsertCosmosDBv5 {
    protected static final Logger log = LoggerFactory.getLogger(ReadJsonInsertCosmosDBv5.class);
    static String databaseName = "testdb";
    static String end_point = "https://cosmosdbtest.documents.azure.com:443";
    static String master_key = "yourkey";
    static String containerName = "colltionname";
    static int k = 0;

    public static class Family {
        public String id;
        public String lastName;
        public boolean isRegistered;
        public String getId() {
            return id;
        }
        public String getLastName() {
            return lastName;
        }
        public Family(String idv, String lastnamev, boolean isv) {
            id = idv;
            lastName = lastnamev;
            isRegistered = isv;
        }
        public Family() {
        }
    }

    public static void main(String... args) {
        log.info("Read the json file");
        JSONParser jsonParser = new JSONParser();
        try (FileReader reader = new FileReader("xxxxx/test1.json")) {
            Object obj = jsonParser.parse(reader);
            JSONArray pListr = (JSONArray) obj;

            try (CosmosClient cosmosClient = new CosmosClientBuilder().endpoint(end_point).key(master_key).directMode(DirectConnectionConfig.getDefaultConfig()).consistencyLevel(ConsistencyLevel.SESSION).connectionSharingAcrossClientsEnabled(true).contentResponseOnWriteEnabled(true).userAgentSuffix("applicationtest").buildClient()) {
                log.info("The connection to Cosmos DB succeeded.");
                CosmosDatabase database = cosmosClient.getDatabase(databaseName);
                log.info("Set item");
                CosmosItemRequestOptions cosmosItemRequestOptions = new CosmosItemRequestOptions();
                log.info("Set Container");
                CosmosContainer container = database.getContainer(containerName);

                log.info("Write the json data into Cosmos DB");
                pListr.forEach(pers -> parsePersonObject((JSONObject) pers, container, cosmosItemRequestOptions));

                log.info("Read data from Cosmos DB");
                log.info("Set some common query options");
                CosmosQueryRequestOptions queryOptions = new CosmosQueryRequestOptions();
                log.info("Set query metrics enabled to get metrics around query executions");
                CosmosPagedIterable<ReadJsonInsertCosmosDBv5.Family> familiesPagedIterable = container.queryItems("select * from c", queryOptions, ReadJsonInsertCosmosDBv5.Family.class);
                familiesPagedIterable.iterableByPage(30).forEach(cosmosItemPropertiesFeedResponse -> {
                    System.out.println("Got results " + cosmosItemPropertiesFeedResponse.getResults().size() + " items(s)");
                    System.out.println("Item Ids " + cosmosItemPropertiesFeedResponse
                            .getResults()
                            .stream()
                            .map(ReadJsonInsertCosmosDBv5.Family::getId)
                            .collect(Collectors.toList())
                    );
                }
                );
            } catch (Exception e) {
                log.info("Something went wrong! ", e);
            } finally {
                if (reader != null) {
                    try {
                        reader.close();
                    } catch (IOException ioe) {
                        ioe.printStackTrace();
                    }
                }
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (ParseException e) {
            e.printStackTrace();
        }
    }

    private static void parsePersonObject (JSONObject person1, CosmosContainer container1, CosmosItemRequestOptions cosmosItemRequestOptions1){
        JSONObject personObject = (JSONObject) person1.get("person");
        String id = (String) person1.get("id");
        boolean isRegistered = (boolean) personObject.get("isRegistered");
        String lastName = (String) personObject.get("LastName");

        Family family = new Family(id, lastName, Boolean.valueOf(isRegistered));
        CosmosItemResponse<Family> item = container1.createItem(family, new PartitionKey(family.getLastName()), cosmosItemRequestOptions1);
    }
}

3-2. 実行結果

[main] INFO ReadJsonInsertCosmosDBv5 - Read the json file
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Initializing DocumentClient [0] with serviceEndpoint [https://cosmosdbtest.documents.azure.com:443], connectionPolicy [ConnectionPolicy{requestTimeout=PT5S, connectionMode=DIRECT, maxConnectionPoolSize=1000, idleHttpConnectionTimeout=null, idleTcpConnectionTimeout=PT0S, userAgentSuffix='applicationtest', throttlingRetryOptions=RetryOptions{maxRetryAttemptsOnThrottledRequests=9, maxRetryWaitTime=PT30S}, endpointDiscoveryEnabled=true, preferredRegions=null, multipleWriteRegionsEnabled=true, proxyType=null, inetSocketProxyAddress=null, readRequestsFallbackEnabled=true, connectTimeout=PT5S, idleTcpEndpointTimeout=PT1H, maxConnectionsPerEndpoint=130, maxRequestsPerConnection=30, tcpConnectionEndpointRediscoveryEnabled=false}], consistencyLevel [Session], directModeProtocol [Tcp]
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - creating a new shared HttpClient
[parallel-1] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Getting database account endpoint from https://cosmosdbtest.documents.azure.com:443
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - creating a new shared RntbdTransportClient
[main] INFO com.azure.cosmos.implementation.directconnectivity.RntbdTransportClient - Using default Direct TCP options: azure.cosmos.directTcp.defaultOptions
[main] INFO com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdServiceEndpoint$RntbdEndpointMonitoringProvider - Starting RntbdClientChannelPoolMonitoringProvider ...
[main] INFO ReadJsonInsertCosmosDBv5 - The connection to Cosmos DB succeeded.
[main] INFO ReadJsonInsertCosmosDBv5 - Set item
[main] INFO ReadJsonInsertCosmosDBv5 - Set Container
[main] INFO ReadJsonInsertCosmosDBv5 - Write the json data into Cosmos DB

[cosmos-rntbd-nio-2-1] INFO com.azure.cosmos.implementation.SessionContainer - Registering a new collection resourceId [abcdefg] in SessionTokens
[main] INFO ReadJsonInsertCosmosDBv5 - Read data from Cosmos DB

[main] INFO ReadJsonInsertCosmosDBv5 - Set some common query options
[main] INFO ReadJsonInsertCosmosDBv5 - Set query metrics enabled to get metrics around query executions
Got results 2 items(s)
Item Ids [7, 8]

[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Attempting to close client 0
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Closing Global Endpoint Manager ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Closing StoreClientFactory ...
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - closing one reference to the shared RntbdTransportClient, the number of remaining references is 0
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - All references to shared RntbdTransportClient are closed. Closing the underlying RntbdTransportClient
[main] INFO com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdServiceEndpoint$RntbdEndpointMonitoringProvider - Shutting down RntbdClientChannelPoolMonitoringProvider ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down reactorHttpClient ...
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - closing one reference to the shared HttpClient, the number of remaining references is 0
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - All references to shared HttpClient are closed. Closing the underlying HttpClient
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down CpuMonitor ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down completed.

▼4. 参考情報

以上です。参考になりましたら幸いです。



コメントを残す

メールアドレスが公開されることはありません。 が付いている欄は必須項目です