Comment on page
"Hello World" for AWS Kinesis
warpstream playground
The
playground
command will start an Agent on localhost
, store all the data for the Agent in your current directory, and sign up for a temporary account for you to play around with.Let's create a stream using the
aws-cli
tool.aws kinesis create-stream \
--no-sign-request \
--endpoint "http://localhost:8080" \
--stream-name helloworld \
--shard-count 1
If you don't receive an error back, your request to create the stream succeeded.
Now let's write a record to the stream. We'll write a record with the partition key "hello" and the data "world". Record data in Kinesis is base64-encoded, which is why the
Data
field in the example command below is base64-encoded.aws kinesis put-records \
--no-sign-request \
--endpoint "http://localhost:8080" \
--records "Data=\"d29ybGQ=\",PartitionKey=hello" \
--stream-name helloworld
You should receive the response as JSON in your terminal that looks something like this:
{
"FailedRecordCount": 0,
"Records": [
{
"SequenceNumber": "0",
"ShardId": "shardId-000000000000"
}
],
"EncryptionType": "NONE"
}
aws kinesis get-shard-iterator \
--no-sign-request \
--endpoint "http://localhost:8080" \
--stream-name helloworld \
--shard-id shardId-000000000000 \
--shard-iterator-type TRIM_HORIZON
Once you've received a shard iterator back from the Agent, you can start using it to read the record you just wrote in the last step.
aws kinesis get-records \
--no-sign-request \
--endpoint "http://localhost:8080" \
--shard-iterator "SHARD_ITERATOR"
Now you should see your record you wrote previously printed to the console. The
PartitionKey
field is "hello" and the Data
field is the same base64-encoded "world" string as before. You can repeat the process of writing another record and running get-records
command again to read and write more records.{
"Records": [
{
"SequenceNumber": "0",
"Data": "d29ybGQ=",
"PartitionKey": "hello",
"EncryptionType": "NONE"
}
],
"NextShardIterator": "CgtoZWxsb3dvcmxkMxgB",
"MillisBehindLatest": 0
}
You don't need to run the
get-shard-iterator
command again. You can re-use the shard iterator value returned after each successful get-records
.And that's it! You've successfully set up a WarpStream Agent Pool to power your Virtual Cluster, created a stream, and processed some data through it.
You can now move on to trying to create a real application which reads or writes from the stream, or connect your existing tools like Flink to WarpStream.