What is the promise of big data? Computers will be better than humans
Big data as a concept has in fact been around longer than computer technology, which would surprise a number of people.
Back in 1944 Wesleyan University Librarian Fremont Rider wrote a paper which estimated American university libraries were doubling in size every sixteen years meaning the Yale Library in 2040 would occupy over 6,000 miles of shelves. This is not big data as most people would know it, but the vast and violent increase in the quantity and variety of information in the Yale library is the same principle.
What you missed in Big Data: The many languages of analytics
Analytics vendors are providing their users with more and more ways to manipulate data in a bid to stand out from the competition. Last week saw Alphabet Inc. join the fray by rolling out a new time-based partitioning feature for BigQuery, its hosted data warehousing service, that makes it possible to run a query only against information from specified dates.
An analyst that wants to examine their firm’s most recent quarterly sales metrics, for instance, could utilize the functionality to avoid the delay of scanning their entire financial database. And the request can be narrowed down even further using subqueries, another new feature that was added to BigQuery last week. It’s part a revamped SQL implementation that Alphabet describes as much more powerful than the syntax customers had to employ before to interact with their information.
Microsoft bets on Apache Spark to power its big data and analytics services
Microsoft today announced that it is making a serious commitment to the open source Apache Spark cluster computing framework.
After dipping its toes into the Spark ecosystem last year, the company today launched a number of Spark-based services out of preview and announced that the on-premises version of R Server for Hadoop (which uses the increasingly popular open source R language for big data analytics and modeling) is now powered by Spark.