Accomplishing the Impossible

"We don't have the time, people, or technology to do this."

That is what I expected to hear from my colleagues when I suggested we explore how an emerging technology could be useful for our business, which is to provide high quality and efficient risk management, clearing and settlement services for the U.S. exchange-listed options industry and the users of these important financial markets.

Instead, my team responded with a non-committal "we will look in to it, but you know I have a day job" type of response, and I thought that would be the end of it.

However, to my pleasant surprise three months later a group of interns from Stanford University, Northwestern University, The Ohio State University, University of Wisconsin-Whitewater, and University of Missouri were in our board room displaying and explaining data analysis using real-time aggregations on one-half a billion rows of data to OCC's CEO Craig Donohue.

With more than 16 million equity options contracts cleared daily, OCC is a complex business with big data problems that require intelligent, hard-working people to solve. With a little bit of innovation, our challenge to safeguard global financial markets is made a little bit easier through the application of emerging technology trends.

We don't profess to have all of the answers, and as I noted above, all of the time necessary to solve for them. So every summer we bring in a few exceptional interns from around the country to work on a variety of different projects. The scope of this particular data management project was audacious: enable real-time data analytics with on-demand aggregations using obfuscated equity options positions comprised of millions of trades.

I used the word "audacious" because throughout my career in technology and financial markets I have seen very knowledgeable teams spend close to a year on similar efforts without achieving measurable results, and I had low expectations of what this team of bright interns could accomplish over a short summer timeframe. What made it more challenging was that this group had no prior experience with equity options, real-time data analytics, nor Amazon Web Services (AWS) offerings. Nor had they worked with each other!

During their summer experience at OCC, the interns were tasked with pursuing AWS certifications, reading multiple books on various topics, and completing the aforementioned project. They approached the problem by looking at an open-source technology called mapD (massively parallel Database), a solution for GPU-accelerated analytics. Fortunately, with public cloud technology, we did not need to have an Nvida Tesla laying around to support their efforts.

The team created a sandbox within AWS comprised of P2 instances for our mapD solution, which leverages GPU clusters for lightning-fast analytics. They loaded obfuscated and encrypted trade data into AWS Redshift, and developed dashboards that would perform on-demand aggregation and reporting of up to a half-billion rows of data in less than a second.

When their project was completed, the team of interns presented their results to Craig Donohue and other members of OCC's management team. The first words from Craig's mouth were, "How little did this cost? When can we get this into production?"

The completion of this project moves us into a production-hardened implementation. Hardening the application involves developing extensive security controls, automating our data feeds, and designing a high availability architecture. Leveraging our experience from this effort and the additional activities to harden the solution is advancing our risk management capabilities and improving the integrity of the financial markets.

This story is a great example of how OCC aligns its goals and values to help attract the next generation of talent to this very important part of the financial services industry. We strive to foster growth, innovation and thought leadership into everything we do at OCC, and by leading and embracing change we can build a high performance culture that makes OCC a destination for the best talent.

David Hoag, SVP & Chief Information Officer - OCC

OCC (Options Clearing Corporation) is the world's largest equity derivatives clearing organization. As a Systemically Important Financial Market Utility (SIFMU) under the Dodd-Frank legislation, one of OCC's goals is to reduce systemic risks, increase transparency and provide capital and collateral efficiencies for the users of the U.S. exchange-listed options markets.

To learn more about OCC's thought leadership on industry issues, visit OCC's Blog.