Introducing Big Data’s cousin Big Services. What you never heard of Big Services? Well Big Services is perhaps a little shy. OK being a little tongue-in-cheek here but services are proliferating very very fast. And the business opportunity is just as big as taming Big Data.

Services are not a new concept; they started as design guidelines to creating reusable services, evolved to Service-Oriented-Architecture (SOA), and the now to API management. What is amazing is that we are in the age of service proliferation – by one measure public services that can be accessed over the internet are doubling every year and that excludes the hundreds of thousands of services inside the enterprise. Add to this the whole new suites of Software-as-a-Service (SaaS) offerings it becomes a nightmare to even know how many services are out there.

The sheer volume of services is one reason we could call them “Big Services”, but it is the complexity of the different service interfaces, issues of access and control, monitoring, lifecycle management, governance and access that make them really Big Headache Services!

Consider SalesForce.com which has one API (well that seems easy enough) but has a 1,000 page document describing how to access it (OK not so easy). Workday’s opposite approach of publishing 100’s of API’s with 6 or 7 versions. Each WSDL and schema is unique, with different data types (which many need translation form one API to the next), different dependencies (ordering of API calls make a big difference to the end results) and different security and access requirements.

In order to keep track of these Big Services (Volume and complexity) is no easy task. There is a need for software to help you do this. The software needs to provide repository services and a level of abstraction that ensures you work at solving the business problem at hand and not the complexity of the services call. Naturally the software needs to take care of error handling (like what to do if the service is unavailable), monitoring (like checking if the API is available and the average time per call), lifecycle management (like which version of the service to call) and a host of other service security and dependency checking needs. It comes down to whether you have time to build this software of buy it from a trusted partner.

Big Services are creating Big Opportunities: Online services such as Google, Facebook, Netflix and eBay are handling billions of application programming interface (API) calls per day and some companies are accounting for billions of dollars in revenue per year via API links to their services, according to John Musser, founder of ProgrammableWeb. Enterprises are finding that exposing their systems to the world as services create amazing opportunities, just like Best Buy who allows shopping and fulfillment via their API’s, or the EPA which serves up hundreds of self-service APIs that handle air quality to UV index monitoring, or 7-Eleven who exposed coffee purchase API information to determine the results of the election. It isn’t a matter of if, but a matter of when you expose your services to the self-service world, but if you don’t you will miss this BIG services opportunity. Naturally exposing internal systems as services creates its own set of Big Services Headache – stopping denial-of-service attacks, securing the service, publishing the WSDL and documenting the interfaces, and providing access to these services across the mobile and web world.

How will you take advantage of this Big Services opportunity, and provide your business with Big returns?

The post Big Data Meet Big Services appeared first on Reality Check.

Graphical analysis of changes to an API

72% of the respondents in a recent survey identified API lifecycle management as a critical capability of API management solutions.  The survey, that we conducted last week, included over 50 IT architects, developers and consultants. 80% of the survey respondents also said that they have either exposed APIs or are planning to do so in near future.

As a refresher from an earlier post – API Management Primer – I shared the three basic capabilities of an API management solution – API Portal, API Gateway, and API lifecycle management. When the audience was asked, “Which capabilities do you see as most critical for an API management solution?” (which allowed multiple choices) the responses were as follows:

• API Portal – 61%
• API Gateway – 61%
• API Lifecycle Management – 72%

Why do these IT leaders think that API lifecycle management is important? To understand this better, let’s consider the personas on either side of an API – the consumers and the providers.

As an API consumer, say a client app developer you have concerns such as –

• Can version updates to the API break my application?
• Is my API provider meeting the promised service level agreements (SLA’s)?
• Is the API provider applying adequate resources to power the API reliably?

Whereas as an API provider, say the API product manager, may have concerns about –

• Is the API exposed useful and something that will be adopted as expected?
• Are we meeting our organization’s standards in terms of API quality?
• What is our versioning strategy? When do we retire our APIs?
• How do we make sure we understand the impact of changes before making them?

API lifecycle management helps API providers address these concerns for both themselves and also their customers i.e. the app developers. Without addressing the API Lifecycle, API providers will have problems attracting developers to adopt their APIs. In other words, API lifecycle management is critical to implementing a successful API strategy.

Okay, now here’s some food for thought. Of the respondents interested in implementing APIs, over 80% of them said that they are doing APIs for mostly internal use or for both internal and external use. Only 18% were implementing APIs exclusively for external use.  How do you explain that? Share your thoughts by leaving a comment.

The post API Lifecycle is Critical for API Management Solutions appeared first on Reality Check.

API Management Demystified

Several weeks ago, I wrote a Primer on API management and I have received tons of great feedback and questions from blog readers and colleagues in the field. It’s great to see people are finding API management especially relevant to their mobile and cloud strategy.  My colleague David Bressler (has written several posts on this topic – see Enterprise API Trifecta) and has experience addressing API management with both IT and business leaders. So we’ve decided to team up in an effort to summarize our experiences on a webinar called, ‘API Management Demystified’.

In this webinar, we will cover why APIs are important to your business, how APIs present new challenges, and the question I’ve gotten many times, what are the essential API management capabilities? During the webinar, we will share how customers have had success getting their API projects started on the right foot. Please join us for live Q &A at the end of the webinar to get your questions answered and comments addressed. And don’t worry, we’ll be emailing all registrants with the recording so even if you can’t make it, be sure to register. If you’d like to submit a question in advance of the webinar, please leave a comment on the blog and we’ll be sure to answer your question during the webinar. We look forward to seeing you there!

The post API Management Demystified appeared first on Reality Check.

Every other customer I talk to wants to leverage mobile as a key business channel. Consider this – more people on earth have access to mobile phones than a toothbrush(1). That makes everyone with a mobile phone a potential target for your business. No wonder that 61% of the CIOs plan to enhance their mobility capability during the next three years(2). And, mobile apps and middleware spending is up by 45% in 2012.(3)

Broadly speaking, the kind of mobile apps that companies are investing in are –

• Business to enterprise to increase worker productivity especially those who are customer-facing
• Business to consumer to improve customer satisfaction & deepen customer loyalty, and
• Government to citizen to provide information at fingertips and better serve the citizens

But implementing great mobile apps is fraught with challenges when it comes to integrating these apps with rest of the enterprise systems. The key issue I hear from our customers is how to provision mobile apps to access core enterprise systems in a way that is secure and maintains the integrity of these internal systems. In addition, there are several other challenges, such as –

• How to offer reliable mobile user engagement given intermittent connectivity
• How to integrate mobile apps with back-end data sources
• How to assert control over deployed apps
• How are our apps being used, how much and on what platforms
• How to offer consistent authentication and authorization for native, hybrid and web apps

To address these challenges, an architecture based on SOA principles has two major layers –

• Service Mediation Layer
  

  SOA based architecture for enterprise mobile apps

• Business Services Layer

The business services layer enables core enterprise systems to deliver capabilities (functionality and data) as reusable, standards-based services. Residing inside the firewall, these services do the heavy lifting. They perform the most complex business tasks that are needed by enterprise-class mobile apps. However, they do not interact directly with the mobile apps. The service mediation layer acts as a proxy to these services managing all interactions between mobile apps and internal core systems. This is the layer where security is enforced, denial of service attacks are repelled, apps usage and performance is tracked, and messages are translated and routed. The service mediation layer provides a clean way to consolidate all quality-of-service requirements, eliminating the need to code these into individual apps.

Bank of Oklahoma app on the iPad

Many of our customers have used this approach to successfully drive their mobile strategy. An excellent example is Bank of Oklahoma Financial. Head-quartered in Tulsa, Oklahoma with full service branches in 8 states, the bank leveraged their SOA infrastructure to deliver retail banking mobile apps in just 4 months! Not only did they experience outstanding system performance but also negligible production issues. Over time they experienced a large chunk of their customers migrating from web to mobile to access their accounts and lowered call volume to their call centers. That’s the kind of value great mobile apps can deliver for your business.

So, what’s your take? Is this the approach you are taking to fend-off mobile integration challenges? I would love to hear from you.

Sources:

1. Main Street Analyst, More Mobile Phones Than Toothbrushes – Infographic, Jan 2013
2. Gartner, Inc., Magic Quadrant for Mobile Application Development Platforms, April 2012
3. Forrester Research, Inc., Forrsights Budgets And Priorities Tracker Survey, Q4 2011 and Q4 2010

The post The Secret Sauce to Great Mobile Apps appeared first on Reality Check.

In this day of high-speed growth and scale, organizations need to be prepared with a plan to keep up with such changes. In a matter of mere days, new hardware infrastructures are prescribed and acquired. Public and private clouds make it even easier to setup such infrastructures. CIOs are adding new SaaS applications by the dozens and retiring existing legacy applications. All these changes mean only one thing for the IT manager – Insomnia! But, it need not be. Here is my one key recommendation to plan for high-speed growth and scale in 2013 –

“Build an application integration strategy TODAY, if you already don’t have one”

Before I go on to explain how you build one, let me quickly tell you why this is important. Without an application integration strategy, you will keep on building point-to-point connections between systems. This only leads to more chaos and any change in this architecture only means high expenses for time and resources. The most successful and highly-performing organizations definitely have a good application integration strategy in place.

There are 5 basic things that you need to consider when thinking of an application integration strategy –

1. Start with the fundamentals – Setup an ICC (Integration Competency Center). Some of you may groan at this suggestion but I have seen this as a highly productive investment of time and effort in building a company’s integration strategy. Without appropriate governance, anyone with an idea will adopt their own integration methodology and soon, you will end up with a similar situation of point-to-point connectivity or even worse, disconnected silos. Ken Vollmer from Forrester has written a fantastic article on how ICCs help solve integration complexity.
2. The best way to connect systems is to disconnect – As silly as it may sound, the best advice you can get on this is to create a decoupled architecture. By tightly binding data and applications silos with one another, you are only complicating the architecture more. A decoupled architecture is one where one application is agnostic of another application’s presence or preference of connectivity. For example, if you have to send a piece of data from your ERP into Siebel, the ERP can fire the message onto an Enterprise Service Bus (ESB) and forget about it. Siebel will receive that message if it has subscribed to that message topic. This architecture is also known as pub-sub or Publish-Subscribe. This is immensely helpful because, tomorrow, if Siebel gets replaced with Salesforce.com (SFDC), no code needs to be rewritten. The ERP will still fire the message the same way. However, SFDC will now subscribe to the message topic now. As is evident, this architecture allows for greater flexibility, scalability and higher developer productivity.
3. If you are not re-using…that would be amusing – Definitely think SOA (Service-Oriented Architecture) when building upon an application integration strategy. SOA allows for maximum re-use when building your integration logic. This means less resources and more available time for other projects.
4. Think canonicals – This is one of my pet peeves that developers or even architects do not think about using canonicals when building upon an integration strategy. Canonicals are fantastic models to be leveraged to create extensible integrations between two or more complex applications. When you are dealing with two or more complex or large data structures, you will see that you are spending a good chunk of time in mapping the same set of fields or writing custom logic to same field that is named in different ways in different data sets (e.g. FirstName, FName, Name, CustomerName etc). Canonical will act as the flat representation of the superset of all data fields in your integrations. Applications will then connect to the canonical instead of creating new data structures every time they need to be connected with a new system. System-level changes can be easily accommodated with this approach.
5. Use design patterns – One solution does not fit all. If you are an architect, you probably already know what I am going to say in this section. I have seen companies with large IT development teams building integrations with absolutely no design pattern in mind. That is so wrong! Design patterns are established blueprints for success. If you have complex data exchange needs, you need to know for sure what design patterns you are going to use. Design patterns help in reducing complexity and in boosting performance of your integrations.

With that, I want to wish you all a healthy and happy new year!! As you read in my other post, 2013 is the year of application integration. So, be prepared and create an integration strategy. Go on! Connect the world now!!

The post Application Integration Strategy for 2013 and beyond appeared first on Reality Check.

Gartner’s Nexus of Forces

What we are seeing among our customer base is that breakthrough innovation happens when applications tap into the synergy created by at least two (not just one) of these computing models and are tied back into their core enterprise systems. For instance –

• Echo Entertainment, which is in the hospitality and gaming industry, combined the power of Big Data and Mobile to increase the quality of its loyalty program signups. It identified ‘high rollers’ playing in the casino to its floor staff in real time using a mobile app that combined live data from the gaming systems with large amounts of historical data in backend databases to perform in-flight analytics.
• Environmental Protection Agency (EPA) maintains vast datasets of highly distributed information around water, air, waste, and natural resources. It combined the power of Big Data and Cloud by creating and hosting a publicly accessible API that made the data accessible to not only its partner state agencies but also external developers.
• Nemours Children’s hospital a leader in pediatric care combined the power of Social and Information to create ‘smart’ patient rooms. A patient’s arrival triggers an event that enables the system to automatically provision the room with the child’s favorite video games and other electronic entertainment. When a caregiver walks into the patient’s room, another event is triggered enabling the system to retrieve their information from the hospital’s HR system and display it on the room’s TV for immediate identification.

The key point and underpinning of these innovative applications is an event enabled service-oriented architecture. It makes sense, if you think about it. To be truly useful, mobile apps require access to the backend systems via a layer of services. Big Data needs services to deliver relevant and timely business data in order to perform useful analytics. Social applications require combining enterprise events with enterprise data and activity streams where data is delivered by data services. And Cloud requires strong service-oriented governance infrastructure both as a provider as well as consumer of services. Monolithic, client-server, 3-tier architectures cannot provide the extensibility, scalability and agility needed for such innovation.

*Source: Gartner, Inc., The Nexus of Forces: Social, Mobile, Cloud and Information, Chris Howard, et al., June, 2012.

The post Innovative Mobile, Cloud, Big Data & Social Applications appeared first on Reality Check.

But why API’s and why now?

Technically an API (Application Programming Interface) simply defines an immutable contract between API consumers and providers. It is the implementation of the API that actually packages data and/or business logic. What’s changed in the last couple of years is the number of target devices for deploying applications. The options are multiplying rapidly, be it web, smartphones, tablets, laptops, or even cars. By 2020, there would be no less than 10 billion mobile devices in the world.

Which leads to the expectation of  a user experience that now widely varies by the user’s needs, preferred devices, and context. It is becoming increasingly difficult for enterprise IT to serve the needs of such a diverse mix of users. That’s where APIs step in. APIs allow any app development shop out there to create apps that can serve the needs of a specific segment of users. It is a “win-win-win” situation – Users get what they want, app developers monetize the apps they build and enterprise IT gets to serve its diverse end users.

In fact, some of the most popular apps, came out of developer hack-a-thons where developers combined APIs from multiple companies to create a highly innovative app. Without publicly available API’s, this would not have been possible. (See another post in this blog, the API trifecta around the topic of innovation and APIs). APIs therefore, provide a distribution channel for a company’s products and services. Increasingly we are seeing evidence of this in industries across the board but especially in sectors such as media, retail, travel and government. As an IT leader if you decide to provision your APIs externally, you would naturally want to manage them.

So what does that mean in terms of the capabilities you need? 

At the highest level, an API management solution needs to include a –

• Developer Portal for developers to discover APIs, understand usage and sign up for access
• API Gateway that secures and mediates the traffic between your APIs and its consumers
• API Lifecycle Management to manage the process of designing, developing, deploying, versioning and retiring APIs

Of course, the above capabilities may be delivered as-a-service or on-premise or some combination.

If you are with me so far, you are probably now asking -

So how’s API management different from SOA Governance?

There’s quite a bit of confusion out there. You are not alone. That’s because there is significant overlap between the capabilities that are needed to manage APIs and govern an SOA. The key difference lies in the objectives of these two initiatives. API management is concerned with managing external APIs where consumers are unknown. SOA Governance, on the other hand, is about managing services with known consumers.  Note that even for SOA the consumers don’t have to be internal. They could be your business partners who are external but known in advance. (And to be sure, external = outside your firewall.) This distinction leads to design and deployment choices that at times masquerade as differences, adding to the confusion. For instance, REST based APIs are much more popular than SOAP ones, but that’s only because REST services are a lot easier to consume.

Is this helpful in furthering your understanding of API management? Let me know what else you would like to know about API Management.

P.S. What’s common between cowboys, railroads and API management? Stumped? See what my colleague wrote a few months ago on this subject.

The post What is API Management – A Brief Primer appeared first on Reality Check.

Driving innovation at enterprises has always been a tremendous challenge.  There’s always hard to extend legacy systems, disparate systems to be integrated and monolithic applications to be changed. Doing so is painstaking and time consuming work, but the business side of the house is nevertheless focused on seeing immediate return on investments. Exacerbating this situation is the fact that there’s one too many ‘best practices’ to go around making it confusing as to what approach is expected to work for you and your organization.

But is there any getting away from this situation?

The stark reality is that in an uncertain global economy, constant innovation is central for businesses that want to survive and stay competitive. Add to the mix the game changing innovation in personal technology that we have witnessed in the last couple of years. There’s an app for virtually everything, smartphones and tablets are ubiquitous and we are increasingly consuming services in the cloud that has transformed entire industries.

Businesses today are looking at exposing internal capabilities and data via APIs as a new distribution channel for their products and services. They want to adopt the latest capabilities quickly with cloud-based solutions for marketing, sales and human resources that are seamlessly integrated with core enterprise systems. And they want new mobile apps supported on both official and personal devices to empower employees and reach their customers.

What does this mean to you, as an IT leader who is responsible for driving innovation?

It’s time for a comprehensive application strategy that can meet today’s demands for innovation and is well positioned to meet ongoing demands in the future. That means your entire application landscape should be built on a foundation that exposes the needed business capabilities to build new applications and allows rapid changes in the future. Taking a service-oriented approach (SOA) to your application architecture is the foundation for a sustainable innovative future. Customers in various industries have taken this approach with great success, here are just a few examples.

• Avnet is one of the world’s largest distributors of electronic components with over $25B in annual revenue and 100,000 customers. Over 70 acquisitions in 20 years that required integrating disparate applications running on multiple platforms created the urgency for IT to innovate. By adopting SOA along with end-to-end governance, they increased reuse going from 800 services to 111 interfaces, invoked 6M times a month! Avnet not only reduced development time by 55% but also significantly accelerated integration of new acquisitions.
• Endo Pharmaceuticals creates high-value branded products that meet the unmet needs of patients along care pathways for pain, urology, oncology and endocrinology. Their challenge was to integrate a cloud-based HR solution – Workday and synchronize data for 1500 employees to no less than 18 downstream systems. The company used an SOA infrastructure to integrate with Workday API using Pub Sub model for consistent distribution of data. They also established a governance framework to track data from system to system and guarantee security of personal information – critical for compliance reasons and key to success of the entire project. In the process, Endo reduced the time to complete the business process by 70% while increasing accuracy and lowering costs.
• BOK Financial Corporation  is a $27 billion regional financial services company based in Tulsa, Oklahoma. BOKF, NA has full-service banks located in eight states while BOSC, Inc., the broker/dealer subsidiary, provides brokerage services in 10 states. They started on their SOA journey in 2007 establishing an SOA governance committee along the way. In 2010, when they were asked to build a mobile solution with a full set of customer features and a high degree of quality, and speed. They completed the entire project in just four months crediting their architectural design for allowing quick development.

What was common in the approach among these very different companies?

These companies indeed had the right building blocks to establish an SOA and accelerate innovation. These building blocks are software tools to build an SOA infrastructure such that they 

• Are easy to adapt and build upon based on open standards to readily integrate with your existing systems and offer easy customization to suit your specific needs.
• Integrate tightly to eliminate duplicate work, keeping various development and deployment artifacts in sync, increasing automation which helps speed up time-to-value.
• Enable strong governance for service visibility, monitoring and metrics, and for ensuring security and other quality-of-service (QoS) requirements.
• Come with world-class expertise that provides methodology and advice customized for your organization and objectives to ensure successful implementation.

These innovative companies are hardly isolated examples. With the right building blocks,  you and your organization can also un-suck excel at innovation!  Just get the right building blocks! Check out this interesting Infographic on this topic.

The post Innovation Sucks Unless… You Have the Right Building Blocks appeared first on Reality Check.

• #PW12 800 services with 111 interfaces with 6m invocations/month leading to 55% reduction in dev time! Tremend
  ous results at Avnet with #SOA
• #PW12 Ayman Taha of Avnet says that SOA is “not a project” but a cultural change.
• #PW12 I see a top Forrester analyst in the room! @Avnet’s session in room 2.
• #PW12 Avnet wins the IT transformation award with #SOA helping streamline development processes. Congrats to Ayman Taha and team!
• #PW12 Rob James, Echo – Innovation isn’t just technology but a perfect storm by combining business, people, ideas, technology!
• #PW12 Echo entertainment wins the Innovation award for “Innovation”. Congrats to Rob James, CTO and his team!
• #SOA is the foundation for mobile first strategy at Echo in Australia. “WebMethods was able to deliver” #PW12
• “Like Starsky needed Hutch and Simon needed Garfunkel, BPM needed SOA.” #AfricanBank Perfect! #PW12
• #PW12 EPA hosts developer challenge where they are invited to build new applications using EPA’s #API – Very cool! #SOA
• #PW12 One of the requirements for EPA’s CentraSite implementation was to be one stop shop for developers #SOA
• #PW12 Karl Heinz talks about #SOA based global integration that DHL implemented for optimizing its business #PW12

The reality is – SOA is alive and well all around the world. Think I’m just dreaming? Then pinch me (figuratively speaking!), by leaving a comment.

The post Why is SOA so Misunderstood? appeared first on Reality Check.

Incentives for Environmental Innovation: The Environmental Protection Agency (EPA), a US federal agency has implemented a public registry called Reusable Component Services, a clearinghouse of over 1000 assets from several different sources. This ‘umbrella’ registry can be searched by developers for API’s that provide access to EPA’s vast data about the environment. And just like tech leaders such as Google, the EPA even hosts competitions that challenge developers to build innovative apps for the environment. Very cool!

Being Loyal Pays Off: Echo Entertainment is a hospitality, dining, nightlife & gaming group in Australia with over 18 million annual visitors. One of their key business objectives is to sign up ‘High Rollers’ visiting the casinos to their loyalty program. In order to achieve their objective, they equipped their employees on the casino floor with a mobile app that identified, in real time, un-carded high value clients who were playing in the vicinity! Under the covers, the mobile app combined live data from the gaming systems with backend data warehouse systems to report the results back to the floor staff. And get this – within the first week of rolling out the app, they signed up 50 high value clients to their loyalty program!

Now is that not considered ‘innovation’, worthy for the boardroom? You tell me.

The post How to Make SOA a Boardroom Discussion appeared first on Reality Check.