Tag: big data

The biggest big data challenges

big data challenges

Credits: https://blogs-images.forbes.com

We all know that Necessity is the mother of invention and we don’t want to stop at any point in our life because it’s in our gene.

The complex business environment in the world made to invent the concept of big data. Nowadays, data and how to use them make the company different from each other and most important to stay in business. For that companies transform as much as data into a meaningful product with data-driven discoveries for the users. Right analytics on data maximise revenue, improve operations and mitigate risks. According to Demirkan and Dal, big data has following six “V” characteristics i.e Volume, Velocity, Variety, Veracity, Variability and Value. The biggest big data challenges are a bit opaque to see.

IDC predict big data revenue sales will increase more than 50% from nearly $122 billion in 2015 to more than $187 billion in 2019. Nearly 73% of companies increase investing on analytics to transform data into gold but 60 percent of them feel that they don’t have the proper tool to get insight from data. Research predicts that half of all big data projects will fail to deliver desired output.

When Gartner asked what the biggest big data challenges were, the responses suggest that while all the companies plan to move ahead with big data projects, they still don’t have a good idea about what they’re doing and why. The second major concern is not establishing data governance and management. Thomas Schutz, SVP, General Manager of Experian Data Quality. says that “The biggest problem organisations face around data management today actually comes from within,” and “Businesses get in their own way by refusing to create a culture around data and not prioritising the proper funding and staffing for data management.”

There are many challenges but data related issues are biggest challenges in big data.


$19 Trillion Tech Industry Heading Our Way( IoT )


The “Internet of Things” is emerging technology and phrase that 87% of people haven’t heard of. The Internet of things (IoT) is the inter-networking of physical devices, vehicles (also referred to as “connected devices” and “smart devices”), buildings, and other items—embedded with electronics, software, sensors, actuators(MEMs), and network connectivity that enable these objects to collect and exchange data via Internet.

We are surprised to know that ATMs were some of the first IoT objects dated back to 1974.
iot growthIoT market has been growing with the parabolic rate that world has never witnessed. IoT could DWARF every technology before it and it’s just begun…Experts estimate that the IoT will consist of almost 50 billion objects by 2020 only in the span of 4 years. It took 40 years to sell 1 billion Personal Computer, 20 years to reach nearly 7 billion cellphone users and 5 years to reach 1 billion tablets. So we can say that the IoT is the future of ALL technology. It’s literally on the pulse of everything shaping the new Internet.

In order to work ever IoT device has a piece of software and sensors also known as MEMs(Micro-Electro-Mechanical Systems). Software and MEMs make IoT sense, think and act. We can say that MEMs are eye and ear of IoT and Big Data is the “fuel” that power the IoT.

What makes IoT come from nowhere?

To answer this question we have to go through different views of tech luminaries. Inventor of the Ethernet, Bob Metcalfe thinks “It’s a media phenomenon. Technologies and standards and products and markets emerge slowly, but then suddenly, chaotically, the media latches on and BOOM!—It’s the year of IoT.” Chief Economist at Google, Hal Varian believes that it has something to do with Moore’s Law: “The price of processors, sensors, and networking has come way down. Since WiFi is now widely deployed, it is relatively easy to add new networked devices to the home and office”. The father of sensors, Janus Bryzek thinks there are multiple factors. First, there is the new version of the Internet Protocol, IPv6, “enabling the almost unlimited number of devices connected to networks.” Another factor is that four major network providers Cisco, IBM, GE and Amazon—have decided “to support IoT with network modification, adding Fog layer and planning to add Swarm layer, facilitating dramatic simplification and cost reduction for network connectivity.” and last but not least new forecasts made IoT be as future king of all technologies. IoT will add trillions to global GDP. “This is the largest growth in the history of humans,” says Bryzek.

feeding the sleeping Who is feeding the sleeping giant?

recent GE survey reveals, 90% of the company implementing IoT as one of their top 3 priorities. In an active move to accommodate new and emerging technological innovation, the UK Government, in their 2015 budget, allocated £40,000,000 towards research into the Internet of things.Warren Buffet has alone invested $13 billion. Barcelona, Spain invested in it and its water system alone saves $58 million annually. Glasgow, Scotland are investing $37 million in this innovation. “DIGIT“(Developing Innovation and Growing the Internet of Things Act ) is the bill representing investing in IoT by American Govt.

IoT in our daily life and in future with limitless opportunities.

iot every where

The London School of Economics said that “The future is now and [this revolution] is going to disrupt most of the traditional industries”.



It will boost productivity and save companies millions likely billions each and every year.

1.Facebook said that this breakthrough innovation has already saved its data centre by 38%.
2. UPS uses IoT to reduces its fuel consumption by 9 million gallons a 31 million Dollar saving.


driverless car3. Driverless cars will generate $1.3 trillion in annual savings in the United States, with over $5.6 trillions of savings worldwide. The number of cars connected to the Internet worldwide will grow more than sixfold to 152 million in 2020 from 23 million in 2013.


4. The global wearable device market has grown 223% in 2015. It exceeds $1.5 billion in 2014, double its value in 2013.

5. Connected Kitchen saves the food and beverage industry as much as 15% annually.

consumer electronic

6. Consumer Electronics M2M connections will top 7 billion in 2023, generating $700 billion in annual revenue. By 2020, there will be over 100 million Internet-connected wireless light bulbs and lamps worldwide up from 2.4 million in 2013.

7. Health Care industry University of California Medical centre has a robotic control pharmacy that has dispensed 350K prescription without making one error.

legal8. Legal System is also using robots. The New York Times reported that the Blackstone Discovery and e-discovery that uses electronic data discovery software. It can analyse 1.5 million documents for less than 100k dollars. A team of paralegal would have charge 2.1 million to review.


9. Fast Food: – At Eatsa, a futuristic San Francisco-based vegetarian fast food restaurant. There are no employees. The Customer uses a touch screen to order their food. The meal is ready in a matter of minutes.


10. Retail Industry: – RFID technology help tagging the product so that it can be tracked. With IoT, RFID tags retailers can expect 99% accuracy in inventory and there will be 2 to 7% increase in sale.

These above examples are just the tip of Ice-Berg. These machines are getting smarter and smarter every day starting to essentially think on their own.

This technology is not just a game changer. It is THE game changer. So get ready… The Internet of Things is here to stay.

5 pillars of data scientist career

data scientist career

data scientist career

  1. Genetically Modified Leadership
  2. Great Power Great Responsibility
  3. Big Picture in Big Data
  4. KISS
  5. Feedback


  1. Genetically Modified Leadership:

Genetically Modified Leadership You and you alone there to guide and lead yourself. To get noticed by the world you have to modify well exactly like GMO food. Tackle weakness and polishing your strengths lead you to greatness. By concurrently being the best mentor and best student you can possibly be, you will bring forth your GM leadership skills. Leadership will remove self-doubt and motivate you with a heightened self-worth. Control and create your own views to be more positive and productive. The daily intensity of the data scientist role can be increasingly stressful because they have data and data is power.

2. Great Power Great Responsibility:
Great Power Great Responsibility
Power and responsibility always exist in parallel. Nowadays data is power and here comes the responsibility. So, calm yourself being the highest priority then and only then other thing goes well.

3. Big Picture in Big Data:
Big Picture in Big Data
In the age of big data, it is essential to converge your environment into manageable points so that you can keep up a big picture perspective.

4. KISS(keep it short and simple):
The KISS principle states that most systems work best if they are kept simple than made complicated. So, keep rules and procedure of any model short and simple so that it will not burn your ass in coming future. And always keep in mind Albert Einstein quotes – “If you can’t explain it simply, you don’t understand it well”. and “Make things as simple as possible, but not simpler.”

5. Feedback:
Here comes the blood of data scientist i.e feedback. After countless efforts, data scientist waited for feedback to come. Feedback acts like oxygen, blood, all things which make you alive every moment. The more feedback the more powerful you and your work will be.