自己投資としてチャレンジしている内容を Blog で公開しています。
今回は JSON ファイルのデータを読み Cosmos DB (SQL API) に入れる方法について紹介します。
▼1. JSONファイルのデータを Cosmos DB (SQL API) に入れる方法
最初に JSON からデータを読むために FileReaderクラスを利用します。読んだデータは JSONParser で構文解析 (parse) して JSONArray に入れ、Cosmos DB へのデータ挿入用に作成した Family のクラスの引数として渡します。Family のクラスのインスタンス family を item として Cosmos DB のコレクション (コンテイナー) に入れています。今回の例では parsePersonObject のメソッドを作成し、その中で JSONArray から抽出した JSONObject の値を Cosmos DB にデータを挿入しています。
▼2. 事前準備
2-1. Cosmos DB トライアルアカウントの作成
以下のサイトから無料で利用できる Azure Cosmos DB を用意します。Azure を利用するために必要な Subscription を用意する必要もありません。
https://azure.microsoft.com/ja-jp/try/cosmosdb/
2-2. pom.xml にて json-simple の maven 依存関係を追記
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>2-3. Azure Cosmos DB Java SDK v4 4.8.0 を利用
pom.xml に以下のバージョンが指定されていることを確認します。
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>参考情報: Azure Cosmos DB Java SDK v4 for Core (SQL) API: release notes and resources
https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-java-v4
2-4. test.json の中身を確認
[{"person":{"isRegistered":true,"LastName":"Travase"},"id":"7"},{"person":{"isRegistered":true,"LastName":"Ken"},"id":"8"}]参考情報: Java JSON ファイルの作成とデータの読み込み No.21
▼3. JSON ファイルのデータを Cosmos DB (SQL API) に書き込むコード
3-1. Java コードの作成
JSON ファイルのデータを読み Cosmos DB (SQL API) の collection (container) に挿入します。最後に挿入したデータ (item) を確認しています。CosmosDBAccountName、DatabaseName、end_point 、master_key、containerName はそれぞれ利用する値に書き換えます。
import com.azure.cosmos.*;
import com.azure.cosmos.models.CosmosItemRequestOptions;
import com.azure.cosmos.models.CosmosItemResponse;
import com.azure.cosmos.models.CosmosQueryRequestOptions;
import com.azure.cosmos.models.PartitionKey;
import com.azure.cosmos.util.CosmosPagedIterable;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.stream.Collectors;
public class ReadJsonInsertCosmosDBv5 {
protected static final Logger log = LoggerFactory.getLogger(ReadJsonInsertCosmosDBv5.class);
static String databaseName = "testdb";
static String end_point = "https://cosmosdbtest.documents.azure.com:443";
static String master_key = "yourkey";
static String containerName = "colltionname";
static int k = 0;
public static class Family {
public String id;
public String lastName;
public boolean isRegistered;
public String getId() {
return id;
}
public String getLastName() {
return lastName;
}
public Family(String idv, String lastnamev, boolean isv) {
id = idv;
lastName = lastnamev;
isRegistered = isv;
}
public Family() {
}
}
public static void main(String... args) {
log.info("Read the json file");
JSONParser jsonParser = new JSONParser();
try (FileReader reader = new FileReader("xxxxx/test1.json")) {
Object obj = jsonParser.parse(reader);
JSONArray pListr = (JSONArray) obj;
try (CosmosClient cosmosClient = new CosmosClientBuilder().endpoint(end_point).key(master_key).directMode(DirectConnectionConfig.getDefaultConfig()).consistencyLevel(ConsistencyLevel.SESSION).connectionSharingAcrossClientsEnabled(true).contentResponseOnWriteEnabled(true).userAgentSuffix("applicationtest").buildClient()) {
log.info("The connection to Cosmos DB succeeded.");
CosmosDatabase database = cosmosClient.getDatabase(databaseName);
log.info("Set item");
CosmosItemRequestOptions cosmosItemRequestOptions = new CosmosItemRequestOptions();
log.info("Set Container");
CosmosContainer container = database.getContainer(containerName);
log.info("Write the json data into Cosmos DB");
pListr.forEach(pers -> parsePersonObject((JSONObject) pers, container, cosmosItemRequestOptions));
log.info("Read data from Cosmos DB");
log.info("Set some common query options");
CosmosQueryRequestOptions queryOptions = new CosmosQueryRequestOptions();
log.info("Set query metrics enabled to get metrics around query executions");
CosmosPagedIterable<ReadJsonInsertCosmosDBv5.Family> familiesPagedIterable = container.queryItems("select * from c", queryOptions, ReadJsonInsertCosmosDBv5.Family.class);
familiesPagedIterable.iterableByPage(30).forEach(cosmosItemPropertiesFeedResponse -> {
System.out.println("Got results " + cosmosItemPropertiesFeedResponse.getResults().size() + " items(s)");
System.out.println("Item Ids " + cosmosItemPropertiesFeedResponse
.getResults()
.stream()
.map(ReadJsonInsertCosmosDBv5.Family::getId)
.collect(Collectors.toList())
);
}
);
} catch (Exception e) {
log.info("Something went wrong! ", e);
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (ParseException e) {
e.printStackTrace();
}
}
private static void parsePersonObject (JSONObject person1, CosmosContainer container1, CosmosItemRequestOptions cosmosItemRequestOptions1){
JSONObject personObject = (JSONObject) person1.get("person");
String id = (String) person1.get("id");
boolean isRegistered = (boolean) personObject.get("isRegistered");
String lastName = (String) personObject.get("LastName");
Family family = new Family(id, lastName, Boolean.valueOf(isRegistered));
CosmosItemResponse<Family> item = container1.createItem(family, new PartitionKey(family.getLastName()), cosmosItemRequestOptions1);
}
}3-2. 実行結果
[main] INFO ReadJsonInsertCosmosDBv5 - Read the json file
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Initializing DocumentClient [0] with serviceEndpoint [https://cosmosdbtest.documents.azure.com:443], connectionPolicy [ConnectionPolicy{requestTimeout=PT5S, connectionMode=DIRECT, maxConnectionPoolSize=1000, idleHttpConnectionTimeout=null, idleTcpConnectionTimeout=PT0S, userAgentSuffix='applicationtest', throttlingRetryOptions=RetryOptions{maxRetryAttemptsOnThrottledRequests=9, maxRetryWaitTime=PT30S}, endpointDiscoveryEnabled=true, preferredRegions=null, multipleWriteRegionsEnabled=true, proxyType=null, inetSocketProxyAddress=null, readRequestsFallbackEnabled=true, connectTimeout=PT5S, idleTcpEndpointTimeout=PT1H, maxConnectionsPerEndpoint=130, maxRequestsPerConnection=30, tcpConnectionEndpointRediscoveryEnabled=false}], consistencyLevel [Session], directModeProtocol [Tcp]
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - creating a new shared HttpClient
[parallel-1] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Getting database account endpoint from https://cosmosdbtest.documents.azure.com:443
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - creating a new shared RntbdTransportClient
[main] INFO com.azure.cosmos.implementation.directconnectivity.RntbdTransportClient - Using default Direct TCP options: azure.cosmos.directTcp.defaultOptions
[main] INFO com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdServiceEndpoint$RntbdEndpointMonitoringProvider - Starting RntbdClientChannelPoolMonitoringProvider ...
[main] INFO ReadJsonInsertCosmosDBv5 - The connection to Cosmos DB succeeded.
[main] INFO ReadJsonInsertCosmosDBv5 - Set item
[main] INFO ReadJsonInsertCosmosDBv5 - Set Container
[main] INFO ReadJsonInsertCosmosDBv5 - Write the json data into Cosmos DB
[cosmos-rntbd-nio-2-1] INFO com.azure.cosmos.implementation.SessionContainer - Registering a new collection resourceId [abcdefg] in SessionTokens
[main] INFO ReadJsonInsertCosmosDBv5 - Read data from Cosmos DB
[main] INFO ReadJsonInsertCosmosDBv5 - Set some common query options
[main] INFO ReadJsonInsertCosmosDBv5 - Set query metrics enabled to get metrics around query executions
Got results 2 items(s)
Item Ids [7, 8]
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Attempting to close client 0
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Closing Global Endpoint Manager ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Closing StoreClientFactory ...
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - closing one reference to the shared RntbdTransportClient, the number of remaining references is 0
[main] INFO com.azure.cosmos.implementation.directconnectivity.SharedTransportClient - All references to shared RntbdTransportClient are closed. Closing the underlying RntbdTransportClient
[main] INFO com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdServiceEndpoint$RntbdEndpointMonitoringProvider - Shutting down RntbdClientChannelPoolMonitoringProvider ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down reactorHttpClient ...
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - closing one reference to the shared HttpClient, the number of remaining references is 0
[main] INFO com.azure.cosmos.implementation.http.SharedGatewayHttpClient - All references to shared HttpClient are closed. Closing the underlying HttpClient
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down CpuMonitor ...
[main] INFO com.azure.cosmos.implementation.RxDocumentClientImpl - Shutting down completed.▼4. 参考情報
- Azure Cosmos DB を無料で試す
- クイックスタート: Azure Cosmos DB for NoSQL のデータを管理するための Java アプリを作成する
- Java JSON ファイルの作成とデータの読み込み No.21
- Java Azure Cosmos DB (SQL API) データの参照方法 No.15
以上です。参考になりましたら幸いです。