Big Data Meet Big Services

The industry is full of the hype surrounding Big Data and it seems that everywhere you go there is another infographic on Big Data. Big Data has some very impressive numbers with data growing at an amazing rate. At last count we are almost at 4 Zettabytes growing to a projected 35 Zettabytes by 2020 I am sure this projection will be low and soon we will be talking about Yottabyte (may the force be with us). BUT, with all this focus on data there is another area that is growing just as fast.

Introducing Big Data’s cousin Big Services. What you never heard of Big Services? Well Big Services is perhaps a little shy. OK being a little tongue-in-cheek here but services are proliferating very very fast. And the business opportunity is just as big as taming Big Data.

Services are not a new concept; they started as design guidelines to creating reusable services, evolved to Service-Oriented-Architecture (SOA), and the now to API management. What is amazing is that we are in the age of service proliferation – by one measure public services that can be accessed over the internet are doubling every year and that excludes the hundreds of thousands of services inside the enterprise. Add to this the whole new suites of Software-as-a-Service (SaaS) offerings it becomes a nightmare to even know how many services are out there.

The sheer volume of services is one reason we could call them “Big Services”, but it is the complexity of the different service interfaces, issues of access and control, monitoring, lifecycle management, governance and access that make them really Big Headache Services!

Consider which has one API (well that seems easy enough) but has a 1,000 page document describing how to access it (OK not so easy). Workday’s opposite approach of publishing 100’s of API’s with 6 or 7 versions. Each WSDL and schema is unique, with different data types (which many need translation form one API to the next), different dependencies (ordering of API calls make a big difference to the end results) and different security and access requirements.

In order to keep track of these Big Services (Volume and complexity) is no easy task. There is a need for software to help you do this. The software needs to provide repository services and a level of abstraction that ensures you work at solving the business problem at hand and not the complexity of the services call. Naturally the software needs to take care of error handling (like what to do if the service is unavailable), monitoring (like checking if the API is available and the average time per call), lifecycle management (like which version of the service to call) and a host of other service security and dependency checking needs. It comes down to whether you have time to build this software of buy it from a trusted partner.

Big Services are creating Big Opportunities: Online services such as Google, Facebook, Netflix and eBay are handling billions of application programming interface (API) calls per day and some companies are accounting for billions of dollars in revenue per year via API links to their services, according to John Musser, founder of ProgrammableWeb. Enterprises are finding that exposing their systems to the world as services create amazing opportunities, just like Best Buy who allows shopping and fulfillment via their API’s, or the EPA which serves up hundreds of self-service APIs that handle air quality to UV index monitoring, or 7-Eleven who exposed coffee purchase API information to determine the results of the election. It isn’t a matter of if, but a matter of when you expose your services to the self-service world, but if you don’t you will miss this BIG services opportunity. Naturally exposing internal systems as services creates its own set of Big Services Headache – stopping denial-of-service attacks, securing the service, publishing the WSDL and documenting the interfaces, and providing access to these services across the mobile and web world.

How will you take advantage of this Big Services opportunity, and provide your business with Big returns?


About Mark Herring

Mark Herring has written 19 posts in this blog.

Technology marketing guy with a passion for positioning and marketing products that exceed customer needs.


Leave a Reply