{"_id":"56c41ca22d97560d00e23cfd","category":{"_id":"56c4183328bd680d005e7ac6","pages":["56c41c94106c12170020dba2","56c41ca22d97560d00e23cfd","56c41cad4f0aed1700afabaa","56c41cb670187b17005f43bd"],"project":"56bc8e679afb8b0d00d62dcf","__v":4,"version":"56bc8e689afb8b0d00d62dd2","sync":{"url":"","isSync":false},"reference":true,"createdAt":"2016-02-17T06:50:27.750Z","from_sync":false,"order":1,"slug":"iot-telemetry","title":"IoT Telemetry"},"parentDoc":null,"version":{"_id":"56bc8e689afb8b0d00d62dd2","project":"56bc8e679afb8b0d00d62dcf","__v":18,"createdAt":"2016-02-11T13:36:40.146Z","releaseDate":"2016-02-11T13:36:40.146Z","categories":["56bc8e689afb8b0d00d62dd3","56c3c837bc41330d009f25ed","56c3c83e521f350d00d348eb","56c3c8452d97560d00e23cd8","56c3c85234df460d00c2beb8","56c4180d70187b17005f43b4","56c418162d97560d00e23cf6","56c4181cc4796b0d007ef039","56c4182370187b17005f43b5","56c418292e75e01700986052","56c4183328bd680d005e7ac6","56c4183bbb64720d00552b88","56c418414040602b0064cea0","56c4184754b6030d00ec29a1","56c4184c28bd680d005e7ac7","56c4185370187b17005f43b6","56c4185b6063071700500cfc","582a98b6f8c0a0190053d7a5"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"__v":2,"project":"56bc8e679afb8b0d00d62dcf","user":"56b98db7bb36440d0001f492","updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-02-17T07:09:22.458Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":1,"body":"### Devices and Data..._Lots of Data_\n\nAs we move into a world where more and more devices are connected (sometimes called the _Internet of Things_), these devices can generate a lot of data. And this data can be very valuable. \n\nBut being a producer or application developer for these devices presents new questions and problems not before considered such as:\n\n* Where do I store all of the data generated by these applications?\n* How do I know how many devices are out there, where they are, and what their usage patterns are?\n* How do I know that my user's data is secure, protected, and stored in accordance with the data storage requirements various countries and locales?\n* Once I have the data, what do I do with it?  How do I use it to make my business smarter and more efficient?\n\n### Introducing Buddy Telemetry \n\nBuddy telemetry is specifically designed to answer these questions.  At it's core, Buddy Telemetry offers three things:\n\n1. All the power of the Buddy Platform for any device that can talk to the Internet\n\n2. The ability to safely store large amounts of device data for later use.  Using Buddy's global reach, you can choose the physical locality for your all of your apps data.\n\n3. The ability to easily manipulate and harvest small amounts of device data for immediate use.\n\nBuddy does all of this natively, so you don't need to spend any time building a backend, worrying about scaling, or building a management portal to monitor your devices and applications.\n\nLet's take an example of a new device called _The Widget_, which is present in customer's homes and connects to the Internet via their home WiFi.  \n\nThe Widget periodically sends back usage information about when it's being used, and with what settings, in a JSON format like the following:\n\n    {\n      serial_number: '12345',\n      user_name: 'someone:::at:::somewhere.com',\n      widget_setting_1: 12,\n      widget_setting_2: \"high\",\n      is_active: true,\n      cpu_usage: 12,\n      memory_usage: 87\n    }\n\nThat may not seem like much data until you think about millions of devices, all generating this data at regular intervals. \n\nBuddy Telemetry will save _all_ of this data for you so you can make use of it when you desire. \n\n#### Configuring Telemetry\n\nThe first thing you'll need to do is [configure telemetry](doc:configure-telemetry). This tells Buddy a little bit about the data you're sending, and how you want it handled. It's an easy operation, and we'll walk through some of it's features below.\n\n##### Protecting Customer Information\n\nThe configuration `filterKeys` parameter allows you to have information removed from the data before storage.\n\nYou may have noticed a customer email in that information. You don't want that going into the data store, so you can set the `filterKeys` to `['user_name']`, which will cause the `user_name` field of any passed in data to be removed before storage. For sub-fields in JSON data, you can specify them like `'info.name'`.\n      \n#### Controlling Data Volume\n\nThe configuration `sampleRate` parameter allows you to control what percentage of data will be captured.\n\nIf you've got millions of devices sending data, you might not want to keep _all_ of the data. By setting the `sampleRate` configuration parameter to a value between `0` and `100`, you can choose what percentage of data samples to store.\n\n#### Realtime Data Analytics\n\nEven though you're storing most of the generated data, you may want to track certain aspects of this data in real time.\n\nBuddy makes this easy. First, you'll get all of the value of the developer dashboard - number of installs, recent usage locations, device types/versions, etc. - just like any other Buddy app.\n\nBut you'll also be able to customize what data is available to you. Buddy supports \"metrics\" which are app-specific markers that are sent to Buddy to track anything the app developer would like, such as a feature usage, or a button click.\n\nWith Buddy Telemetry, you can get the best of both worlds by defining values in the data stream that will automatically be converted into Metrics events, with the `metricsKeys` parameter. This parameter instructs Buddy Telemetry to convert the specified keys into metrics events.\n\nFor example, if `metricsKeys` was specified as `['widget_setting_1', 'is_active']`, when device data came in, Buddy would add two metrics events with the key/value pairs as `widget_setting_1 / 12` and `is_active / true`, respectively.\n\n#### Using Telemetry\n\nUsing telemetry is a two-step process for your app:\n\n1. Create the telemetry configuration, specifying how you want Buddy Telemetry to process the data\n\n2. Sending data to Buddy Telemetry\n\nIf you'd like to ensure that the data is being processed correctly, also check out the Test Telemetry operation.","excerpt":"","slug":"telemetry-overview","type":"basic","title":"Telemetry Overview"}

Telemetry Overview


### Devices and Data..._Lots of Data_ As we move into a world where more and more devices are connected (sometimes called the _Internet of Things_), these devices can generate a lot of data. And this data can be very valuable. But being a producer or application developer for these devices presents new questions and problems not before considered such as: * Where do I store all of the data generated by these applications? * How do I know how many devices are out there, where they are, and what their usage patterns are? * How do I know that my user's data is secure, protected, and stored in accordance with the data storage requirements various countries and locales? * Once I have the data, what do I do with it? How do I use it to make my business smarter and more efficient? ### Introducing Buddy Telemetry Buddy telemetry is specifically designed to answer these questions. At it's core, Buddy Telemetry offers three things: 1. All the power of the Buddy Platform for any device that can talk to the Internet 2. The ability to safely store large amounts of device data for later use. Using Buddy's global reach, you can choose the physical locality for your all of your apps data. 3. The ability to easily manipulate and harvest small amounts of device data for immediate use. Buddy does all of this natively, so you don't need to spend any time building a backend, worrying about scaling, or building a management portal to monitor your devices and applications. Let's take an example of a new device called _The Widget_, which is present in customer's homes and connects to the Internet via their home WiFi. The Widget periodically sends back usage information about when it's being used, and with what settings, in a JSON format like the following: { serial_number: '12345', user_name: '[email protected]', widget_setting_1: 12, widget_setting_2: "high", is_active: true, cpu_usage: 12, memory_usage: 87 } That may not seem like much data until you think about millions of devices, all generating this data at regular intervals. Buddy Telemetry will save _all_ of this data for you so you can make use of it when you desire. #### Configuring Telemetry The first thing you'll need to do is [configure telemetry](doc:configure-telemetry). This tells Buddy a little bit about the data you're sending, and how you want it handled. It's an easy operation, and we'll walk through some of it's features below. ##### Protecting Customer Information The configuration `filterKeys` parameter allows you to have information removed from the data before storage. You may have noticed a customer email in that information. You don't want that going into the data store, so you can set the `filterKeys` to `['user_name']`, which will cause the `user_name` field of any passed in data to be removed before storage. For sub-fields in JSON data, you can specify them like `'info.name'`. #### Controlling Data Volume The configuration `sampleRate` parameter allows you to control what percentage of data will be captured. If you've got millions of devices sending data, you might not want to keep _all_ of the data. By setting the `sampleRate` configuration parameter to a value between `0` and `100`, you can choose what percentage of data samples to store. #### Realtime Data Analytics Even though you're storing most of the generated data, you may want to track certain aspects of this data in real time. Buddy makes this easy. First, you'll get all of the value of the developer dashboard - number of installs, recent usage locations, device types/versions, etc. - just like any other Buddy app. But you'll also be able to customize what data is available to you. Buddy supports "metrics" which are app-specific markers that are sent to Buddy to track anything the app developer would like, such as a feature usage, or a button click. With Buddy Telemetry, you can get the best of both worlds by defining values in the data stream that will automatically be converted into Metrics events, with the `metricsKeys` parameter. This parameter instructs Buddy Telemetry to convert the specified keys into metrics events. For example, if `metricsKeys` was specified as `['widget_setting_1', 'is_active']`, when device data came in, Buddy would add two metrics events with the key/value pairs as `widget_setting_1 / 12` and `is_active / true`, respectively. #### Using Telemetry Using telemetry is a two-step process for your app: 1. Create the telemetry configuration, specifying how you want Buddy Telemetry to process the data 2. Sending data to Buddy Telemetry If you'd like to ensure that the data is being processed correctly, also check out the Test Telemetry operation.