How the Startup Ping++ Chose Its Big Data Platform
How the Startup Ping++ Chose Its Big Data Platform
We look at how one startup used a big data platform provided by a key player in the space to gain insights and build services faster.
Join the DZone community and get the full member experience.Join For Free
Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.
Ping++ is an aggregated payment company under Shanghai Jianmi Network Technology Co., Ltd that has grown into one of China's leading third-party payment solution SaaS service providers. Founded in April 2014, Ping++ gained USD $10 million in series B funding from China Broadband Capital at the beginning of 2016. Ping++ provides more than 15,000 merchants in sectors like retail, e-commerce, O2O, education, tourism, and SaaS service with customized payment solutions.
Ping++ offers a secure and reliable aggregation payment platform to the straightforward and worry-free aggregation payment cloud service. It significantly simplifies the process of connecting enterprises, mobile payment channels, and installment payment channels. It does so by integrating functions such as daily transaction management, accounting management, Big Data analytics, multi-level merchant, and user accounts on a unified management platform. From payment access, transaction processing, and business analysis to business operations and Big Data marketing, Ping++ builds customized end-to-end solutions to help enterprises rise to the challenges posed by commercialization.
Overcoming the Challenges of Massive Transaction Volumes
Ping++ provides a unified SDK interface to its cloud payment processing system that allows merchants to make transaction requests through third-party payment channels. The merchant can initiate the transaction by calling an SDK interface, storing records including payment time, amount, and method in a database. Transaction volume in today's word is massive, often reaching millions of transactions a day. One can answer the questions of how to enable customers through data, adding value, and improving customer loyalty in two words: Big Data. Thus, Ping++ recently found itself in urgent need of a stable, reliable Big Data platform. It had to overcome several challenges when setting up its Big Data platform.
While a Big Data platform can help reduce the cost of data innovation and startup operational costs, it needs to provide multiple functions such as storage, computing, business intelligence (BI), and machine learning.
- Storage: A data warehouse is used to store massive amounts of payment order data.
- Computing: ETL data cleaning, statistical summary, and data analysis are all performed in mass.
- BI: Data monitoring indicators are generated to display data quality and trends visually.
- Machine Learning: Models are trained and forecast for every situation.
As a startup organization, Ping++ tries its best to create business innovation at the lowest cost possible. However, to set up a Big Data platform that meets these four requirements, Ping++ had to invest a significant amount in hardware, time, and manpower. As an Internet company, Ping++ needs to grow fast. It is of critical importance that the company is able to set up a Big Data platform quickly, efficiently, and economically.
Security is an unavoidable topic in the financial industry. As an Internet payment service provider, Ping++ attaches tremendous importance to the security, stability, and reliability of its Big Data platform. Additionally, it requires the platform to have a multi-role and multi-privilege account system. Meanwhile, Ping++ also uses a strict data privacy protection policy that ensures that each merchant's data is only used to analyze its own business.
How Ping++ Chose Its a Public Cloud Service Provider
As mentioned above, building a Big Data platform that meets all of the requirements can be costly and time-consuming. As Ping++ is itself a third-party service provider and adheres to the philosophy that professional personnel do professional things, it hoped to solve the issue through other third-party services. Therefore, Ping++ planned to use a public cloud Big Data platform from the very beginning. "We believe that a new company with a small team is often better off taking advantage of cloud services to save time and money. It allows us to spend more of our energy on developing great applications and service,." said Xia Sumin, Big Data Director of Ping++.
The Big Data technical team at Ping++ experimented and compared several mainstream public cloud Big Data platforms. In particular, it focused on platforms provided by large international cloud computing companies and found that the services provided by Chinese organizations are generally not up to the mark and less stable than others. However, if Ping++'s entire transaction system is on Alibaba Cloud, Ping++ can be certain of its security and stability. Furthermore, if the Big Data platform uses Alibaba Cloud to transmit data, then compatibility and cost also see vast improvements. In view of the considerations, Ping++ finally selected the Alibaba Cloud Big Data platform.
Ping++'s Solution and Architecture
Ping++ was able to make the following innovations by analyzing swaths of transaction data on Alibaba Cloud's Big Data platform:
BI Marketing Business System: Ping++ began by drawing a portrait of each user based on their transaction behavior. Based on these user portraits, it developed a precision marketing plan with the goal of regaining old customers, raising customer prices, acquiring new customers, and increasing conversion rates.
Internal Monitoring: Using the Alibaba Cloud Big Data platform, Ping++ successfully managed data including transaction information summaries, alerts for abnormal transactions, and data quality analysis.
Data source: The source of the data is the payment system, which relies on Alibaba cloud RDS, DRDS, and MongoDB. The solution uses DRDS and RDS to store users' transaction information. Additionally, it uses MongoDB to store information on different product dimensions.
Big Data Platform: The Big Data platform is based on the Alibaba Cloud Big Data platform and includes the following components:
- MaxCompute (formerly ODPS): Ping++ uses MaxCompute as its data warehouse. The current data warehouse includes more than 630 data tables, including the original base table, ETL results table, and upper-layer ADM application data market table.
- DataIDE: In DataIDE, the DAG scheduling system is used to support more than 140 nodes that run routinely every day. This provides logging, alarm functions, and data supplementation to facilitate the daily operation and maintenance of the system and its data. The Alibaba Cloud sub-accounts and Big Data platform constitute a permission system that is driven by a multi-role, multi-platform model.
- Quick BI: It allows users to create graphic representations of different indicators. Quick BI offers over 10 different charts to aid data quality monitoring and decision making.
- Machine Learning: Its in-built algorithm library simplifies the graphical modeling of large data sets, machine learning classifications, and text segmentation.
- Storing and Retrieving Results: Data smart tables stored in MaxCompute are primarily comprised of summarized, desensitized data. The main reason for using AnalyticDB is that it possesses powerful computing capabilities that enable the real-time analysis of retrieved data. A MySQL database stores forecast and analysis results garnered via machine learning for retrieval analysis.
- Application: BI marketing system and internal monitoring.
How Alibaba Cloud Big Data Helped Ping++
One-stop Big Data Solution: Alibaba Cloud Big Data platform provides Ping++ with a complete set of one-stop Big Data solutions. It offers enterprise data warehouses, BI, machine learning, and data visualization that help make Ping++ more agile, smarter, and more insightful. Alibaba Cloud Big Data platform features an integrated development environment that covers the entire Big Data chain. This includes data importing, search, development, ETL, scheduling, deployment, modeling, BI reporting, and machine learning to service development, release, and external data exchange. The solution helped Ping++ to avoid having to reinvent the wheel simply to solve problems that the entire industry is facing.
Minimizing Costs: Through the Alibaba Cloud Big Data platform, Ping++ has been able to quickly erect a Big Data platform at a significantly reduced cost. This has given Ping++ the liberty to direct more manpower and material resources to innovate their core services. "We have already begun offering our Big Data services. If we had built the Big Data platform ourselves, we would have only just now finished it, and would not have even begun building our actual business. In short, this platform has saved us 6 to 8 months of work," said Xia Sumin.
Enhanced Security: The Alibaba Cloud Big Data platform provides security that meets the standards of financial enterprises, guaranteeing that Ping++ is able to remain secure, dependable, and stable. The core product, MaxCompute, provides multiple security mechanisms to protect Ping++'s data. For example, it uses the industry standard API authentication protocol, which does not grant super administrator permissions. Additionally, it provides rich user management and authorization functionality in the project space, isolates user operations, and uses the principle of minimum permission, data access auditing, and security monitoring systems throughout the operation. "We once suffered a DDoS attack; however, Alibaba Cloud was able to help us successfully defend it," Xia Sumin recalled.
Published at DZone with permission of Leona Zhang . See the original article here.
Opinions expressed by DZone contributors are their own.