{"id":1362,"date":"2014-06-20T07:52:38","date_gmt":"2014-06-20T14:52:38","guid":{"rendered":"http:\/\/www.robinstewart.com\/blog\/?p=1362"},"modified":"2014-06-20T07:52:38","modified_gmt":"2014-06-20T14:52:38","slug":"metadata-visualization","status":"publish","type":"post","link":"https:\/\/www.robinstewart.com\/blog\/2014\/06\/metadata-visualization\/","title":{"rendered":"Metadata Visualization"},"content":{"rendered":"<p>There are at least two ways of interpreting a table of data.<\/p>\n<table border=\"1\" cellspacing=\"0\" cellpadding=\"5\">\n<tbody>\n<tr>\n<td>Date<\/td>\n<td>Temperature<\/td>\n<td>Humidity<\/td>\n<\/tr>\n<tr>\n<td>June 18<\/td>\n<td>92<\/td>\n<td>57<\/td>\n<\/tr>\n<tr>\n<td>June 19<\/td>\n<td>95<\/td>\n<td>NULL<\/td>\n<\/tr>\n<tr>\n<td>June 20<\/td>\n<td>84<\/td>\n<td>51<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The first\u00a0interpretation treats the table\u00a0as a collection of facts about the world. For example, on June 18 the temperature was 92 degrees and the humidity was 57%. On June 19, the temperature was 95 degrees and\u00a0humidity was unknown.<\/p>\n<p>The second interpretation treats the table\u00a0as a literal\u00a0list of\u00a0data points. For example, on June 18, someone recorded the temperature at 92 degrees and the humidity at 57%. On June 19, the humidity sensor was broken. The data is stored in a table with three columns. Before June 18, the\u00a0data was being\u00a0recorded in a different table.<\/p>\n<p>In other words, we\u00a0can focus on what the data\u00a0<em>says<\/em> about the world, or we can focus on the data\u00a0<em>itself<\/em>.<\/p>\n<p>We can think of the data ephemerally as\u00a0<em>information<\/em>, or we can think of it as a physical thing that exists\u00a0in and of itself.<\/p>\n<p>This is analogous to\u00a0written language: a sentence or paragraph generally\u00a0<em>means<\/em> something, but it also exists as\u00a0physical letters and punctuation on the page.<\/p>\n<p>The second interpretation is often called\u00a0<em>metadata:<\/em>\u00a0data about the data. How was it collected, by whom, for what purpose, and where and how is it stored? How accurate is it likely to be?<\/p>\n<p>If we are very confident about the accuracy and relevance of the data, we can summarize and visualize it cleanly. We could show a line chart of temperature over time and start to draw conclusions about what the temperature trend\u00a0<em>means<\/em>.<\/p>\n<p>But if the accuracy and relevance is unknown, we need to take steps to better understand the metadata. How much data is there? Which parts are missing, or appear to be duplicated? Where did it come from? What metrics are most relevant?<\/p>\n<p>Suppose the default behavior of\u00a0a data analysis tool is to ingest\u00a0your data and take you directly to a clean line chart. Is that convenient or misleading? Does that clean line chart imply that you are looking at\u00a0<em>truth,<\/em> when in fact you may just be\u00a0looking at\u00a0<em>data<\/em>?<\/p>\n<p>Can we assume that the line chart is\u00a0about\u00a0<em>temperature<\/em>, or should we\u00a0emphasize\u00a0that it shows\u00a0<em>data about temperature<\/em>? What is the best\u00a0way to communicate that\u00a0distinction?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There are at least two ways of interpreting a table of data. Date Temperature Humidity June 18 92 57 June 19 95 NULL June 20 84 51 The first\u00a0interpretation treats the table\u00a0as a collection of facts about the world. For example, on June 18 the temperature was 92 degrees and the humidity was 57%. On &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.robinstewart.com\/blog\/2014\/06\/metadata-visualization\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Metadata Visualization&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/posts\/1362"}],"collection":[{"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/comments?post=1362"}],"version-history":[{"count":10,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/posts\/1362\/revisions"}],"predecessor-version":[{"id":1379,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/posts\/1362\/revisions\/1379"}],"wp:attachment":[{"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/media?parent=1362"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/categories?post=1362"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.robinstewart.com\/blog\/wp-json\/wp\/v2\/tags?post=1362"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}