IEEE DataPort™ is an easily accessible repository of datasets, including Big Data datasets, and data analysis tools. IEEE DataPort™ is designed to make storage of datasets easier, provide access to valuable datasets across a broad scope of topics, facilitate analysis of datasets, and retain referenceable data for reproducible research.
3 Industries That Will Be Transformed By AI, Machine Learning And Big Data In The Next Decade
Historically, when new technologies become easier to use, they transform industries.
That’s what’s happening with artificial intelligence and big data; as the barriers to implementation disappear (cost, computing power, etc.), more and more industries will put the technologies into use, and more and more startups will appear with new ideas of how to disrupt the status quo with these technologies.
NEBDIH Data Sharing Workshop a Success: “I wish I could have gone to this workshop two years ago!”
On September 29th and 30th, stakeholders from across the Northeast and beyond joined the Hub at Drexel University in Philadelphia for “Enabling Seamless Data Sharing in Industry and Academia,” a cross-sector workshop put on by our community to tackle the challenges of sharing data head-on. In short “TED talk”-style presentations and collaborative breakout sessions, participants from industry, academia, government and non-profit organizations articulated the challenges they’ve encountered in sharing data; presented real-life case studies of data sharing failures and successes; identified lessons learned and best practices; and dreamed big on possible solutions to these challenges, both technical and non-technical.
Researchers to Study the Big Data Hubs as an Emerging Data Ecosystem
We don’t just facilitate innovative research at the #BDHubs – we go under the lens ourselves! A new study will examine the Hubs’ ongoing partnership-building activities, to better understand the rise and institutionalization of data sciences from the Hubs’ unique juncture of the academic, industry, government and non-profit sectors. A full announcement from the principal investigators is provided below.
This Mathematician Says Big Data Is Causing a ‘Silent Financial Crisis’
When there is wrongdoing in fields that are both complex and opaque, it often takes a whistle-blower to inform the public. That’s exactly what former quant trader turned social activist Cathy O’Neil has become for the world of Big Data. A Harvard trained mathematician, O’Neil spent the last several years teaching at Barnard, working for DE Shaw, one of the world’s leading hedge funds, and launching a technology start up designed to deliver targeted advertising. Her key takeaway from the last two experiences—that Big Data is increasing inequality and threatening democracy—is the subject of her important new book, Weapons of Math Destruction, out on September 6.
From big oil to big data: Inside Mukesh Ambani's $20-billion startup called Jio
MUMBAI: At the vast open-plan headquarters of Indian telecoms startup Jio, billionaire oil tycoon Mukesh Ambani stands in short sleeves beneath a digital tracker that logs every new subscriber to his service.
The 59-year-old is India's richest man, and his Reliance IndustriesBSE -0.73 % oil & gas group is the country's most profitable.
Now, though, he's betting at least $20 billion on building, from scratch, a national digital empire stretching from phones and hardware to home entertainment and custom-made apps.
Is the Cloud Finally Ready for Big Data?
Few things in technology outside of catching Pikachu on your iPhone have been hyped as much as big data. However, with any trending topic, new technology or invention that’s being called the “wave of the future,” it’s important to look past the hype and analyze what’s actually going on. Outside of widespread media attention and the increasing importance placed on collecting data, big data is rarely used.
According to MIT Technology Review, only 0.5% of digital data is ever analyzed. From a business perspective, this means that millions of companies are losing significant opportunities to increase efficiency, lower costs, and target new customers because they aren’t using the data they’re collecting.
What is the promise of big data? Computers will be better than humans
Big data as a concept has in fact been around longer than computer technology, which would surprise a number of people.
Back in 1944 Wesleyan University Librarian Fremont Rider wrote a paper which estimated American university libraries were doubling in size every sixteen years meaning the Yale Library in 2040 would occupy over 6,000 miles of shelves. This is not big data as most people would know it, but the vast and violent increase in the quantity and variety of information in the Yale library is the same principle.
What you missed in Big Data: The many languages of analytics
Analytics vendors are providing their users with more and more ways to manipulate data in a bid to stand out from the competition. Last week saw Alphabet Inc. join the fray by rolling out a new time-based partitioning feature for BigQuery, its hosted data warehousing service, that makes it possible to run a query only against information from specified dates.
An analyst that wants to examine their firm’s most recent quarterly sales metrics, for instance, could utilize the functionality to avoid the delay of scanning their entire financial database. And the request can be narrowed down even further using subqueries, another new feature that was added to BigQuery last week. It's part a revamped SQL implementation that Alphabet describes as much more powerful than the syntax customers had to employ before to interact with their information.
Microsoft bets on Apache Spark to power its big data and analytics services
Microsoft today announced that it is making a serious commitment to the open source Apache Spark cluster computing framework.
After dipping its toes into the Spark ecosystem last year, the company today launched a number of Spark-based services out of preview and announced that the on-premises version of R Server for Hadoop (which uses the increasingly popular open source R language for big data analytics and modeling) is now powered by Spark.
After Moore's Law: Predicting The Future Beyond Silicon Chips
For several decades now, Georgia Tech professor Tom Conte has been studying how to improve computers: "How do we make them faster and more efficient next time around versus what we just made?"
And for decades, the principle guiding much of the innovation in computing has been Moore's law — a prediction, made by Intel co-founder Gordon Moore, that the number of transistors on a microprocessor chip would double every two years or so. What it's come to represent is an expectation, as The New York Times puts it, that "engineers would always find a way to make the components on computer chips smaller, faster and cheaper."
How Big Data And Analytics Are Transforming Supply Chain Management
Supply chain management is a field where Big Data and analytics have obvious applications. Until recently, however, businesses have been less quick to implement big data analytics in supply chain management than in other areas of operation such as marketing or manufacturing.
Volume, velocity, and variety: Understanding the three V's of big data
We practitioners of the technological arts have a tendency to use specialized jargon. That's not unusual. Most guilds, priesthoods, and professions have had their own style of communication, either for convenience or to establish a sense of exclusivity. In technology, we also tend to attach very simple buzzwords to very complex topics, and then expect the rest of the world to go along for the ride.
Take, for example, the tag team of "cloud" and "big data." The term "cloud" came about because we systems engineers used to draw network diagrams of local area networks. Between the LANs, we'd draw a cloud-like jumble meant to refer to, pretty much, "the undefined stuff in between." Of course, the Internet became the ultimate undefined stuff in between, and the cloud became The Cloud.
Water is life. And, unlike resources such as oil, there is no alternative once it runs out. If we are going to safeguard the world's water supply, then we have to understand and manage every aspect of it. The preciousness of water should mean that wasting it is a crime, as is not fully exploiting information communications technologies (ICT) to intelligently ensure water security for the whole world.
Water is a limited resource, but recent advances in science and technology, and, in particular big data and the Internet of Things (IoT), provide opportunities to prevent its waste. By gathering detailed measurements and leveraging analytics, it is possible to develop an end-to-end picture of our water supply to safeguard its quality and security.
Big Data and the Cloud
The era of big data is well and truly upon us, and it’s no longer a question of whether businesses should engage with big data, as it becomes more and more difficult to gain a competitive advantage if you are not engaged with big data. The volume of data being produced is staggering and will only increase in the near future. Technology giant Cisco predicts that the amount of data produced in 2020 will be 50 times what it is today. No wonder then that companies feel overwhelmed and desperately in need solid advice from specialists who understand their business and have their best interests at heart.
The Big Data Difference: Smart Medical Devices
The future of healthcare is already here. From automated insulin pumps to diagnostic instruments that can interpret their own results, today’s medical devices are smarter and more sophisticated than ever. What’s driving these innovations? The answer is data.
X-rays, MRIs, and ultrasounds produce diagnostic images that let doctors identify abnormalities that can’t be seen from the outside. But what if the same machines that took the images could also interpret the results?
Gartner's BI Reboot, Everybody Loves Spark: Big Data Roundup
Gartner has ripped the guts out of its Magic Quandrant for BI and Analytics Platforms. Quarterly earnings reported by Hortonworks revealed strong revenue growth. BlueData announced new support for streaming platforms, including Apache Spark. And, we get an inside view into the big data workings at eBay. We have all this and more in this week's Big Data Roundup. Plus, we take a look at which Valentine keywords make direct-marketing emails most successful.