How does the hottest Penguin use openstack

  • Detail

How does "big penguin" use openstack

Tencent in Shenzhen is one of the largest technology companies in Asia. The company's motto is to connect more people for a better future and will soon bring popular games to the United States and Europe. It uses openstack on a large scale: there are 14 clusters in 7 data centers in 4 regions. It has been running in production for 4 years, and the availability is 99.99%

superuser interviewed a Tencent team to learn more about Tencent's private cloud, tstack and future plans

who is using openstack

in 2013, Tencent began to build a private cloud called tstack to provide cloud computing resources for internal IT departments. Tsstack provides openstack based infrastructure as a service (IAAs) to OA system operation and maintenance personnel, platform application developers, QA developers and Tencent AI researchers

Tencent's internal IT system, functional department business and most development and quality assurance systems are mainly built in tstack. Tencent has also provided tstack and management experience to many government departments and public enterprises in China, including police departments and power companies

why did Tencent choose openstack

in the early stage, tstack was an infrastructure designed by Tencent and successfully managed more than 6000 Xen virtual machines. However, the original tstack was not a cloud management platform that supported heterogeneous virtualization. This makes it impossible to manage and make full use of many resources, including heterogeneous virtual machines, thousands of physical servers, and many third-party storage devices

in 2014, openstack began to develop rapidly, creating a great ecosystem. Dozens of industry leaders from around the world have participated in openstack and deployed many projects

openstack has become a powerful force in cloud computing and the first choice for open source cloud computing platforms. As the most effective solution for software definition infrastructure, openstack has many advantages, such as open source and advanced design. According to the evaluation and test results of the internal IT operation team, Tencent decided to introduce openstack as the infrastructure of tsstack and expected to provide better services

tsstack is a scalable, high availability and openstack based cloud computing management solution. It provides for managing computing resources storage "For example, the works composed of waste materials are mostly service interfaces of plastic, network, image, authentication and measurement, and are compatible with heterogeneous virtualization, servers, storage devices and network devices. It is also suitable for distributed computing and storage.

the workload running on openstack and its value and impact on Tencent?

tstack is designed for large-scale environments. It manages more than 10000 operating systems System, 40% of which are deployed for more than 300 internal IT services, including OA certification, wechat customs, RTX, mail system, video monitoring, internal security, function management and ERP. These services require 24/7 running time. Tstack also manages the development and testing services of Tencent's various products, such as wechat, browser, games, etc

at the same time, Tencent has also cooperated with the Chinese government to establish an e-government system to manage all aspects of public services such as public transport, taxation, social insurance, medical care, etc. Tstack helps to standardize the interfaces of geographically dispersed data centers and schedule heterogeneous resources. It can reduce server costs by 30% and operation costs by 55%

can you give me some details about deployment

now, tsstack uses openstack as IAAs. It is based on the ILO version of openstack K for the correctness of its data. It has three products: infrastructure cloud, monitoring cloud and self-service cloud. Its main features are heterogeneous cloud management and hybrid cloud management. Infrastructure cloud management physical resources; Self service cloud provides resource application management dashboard, workflow management, service management, PAAS, SaaS and other cloud services; The monitoring cloud provides monitoring and resource utilization for virtual machines, and also implements ITIL processes

tstack uses the following openstack components: Keystone, Nova, neutron, cinder, grace, ionic, heat, swift, Manila, and horizon. Whether to upgrade to the latest openstack version to meet business needs is under evaluation. So far, tstack has deployed 14 clusters in 7 data centers in 4 regions: Shanghai, Chengdu, Tianjin and Shanwei. It manages more than 10000 operating systems and hosts internal IT systems, functional department business systems and most of the development and testing systems within Tencent. It has been used in production for 4 years and its availability exceeds 99.99%

based on the disaster recovery requirements of application services, tstack adopts multi region, dual location and three center deployment. Self service cloud is used for cross region scheduling and management. In a region, three control nodes are used to achieve high availability of keepalive + haproxy. The Galera cluster of MariaDB is a database. Because of its simplicity and efficiency, VLAN is used in most regions, and vxlan is also used in some regions. With openstack, you can manage more than 1000 computing section experiment start points in a single region

openstack has any challenges and how to overcome them

Tencent's journey to openstack was not smooth. Since the tstack team started using tstack four years ago, they have encountered many technical problems. Fortunately, they have always provided stable services for users and overcome these difficulties

here are some typical examples:

nova cannot apply resource quotas to virtual machines that are already running. Therefore, the tstack team has developed the resource quota application function based on Nova open API to apply the resource quota without restarting the virtual machine, especially in the case of limited resources

nova's native resizing feature will restart the virtual machine. The tstack team used the smart ball between the inner and outer barrels of Haier's "no cleaning" washing machine to adopt the special flexible cleaning material ova live resize made of antibacterial high molecular materials to expand this function and adjust the size of cloud instances

although openstack provides quite a number of scheduling strategies based on CPU, memory, disk and node status, it still cannot meet the complex business needs of Tencent. Therefore, tstack customizes scheduling strategies for different services, and realizes high availability by moving virtual machines between hosts

nova block migration will replicate the disk volume. For virtual machines with large volumes, migration is frequently interrupted. The tstack team implemented dynamic adaptive compression during the migration, which greatly saved bandwidth and reduced the migration duration by an average of 50%

heat cannot orchestrate virtual machines created before orchestrating. Tstack team combines the blueking platform in Tencent with heat to realize the full life cycle management of all virtual machines


tstack team developed the neutron plug-in to manage SDN controllers from different vendors


tstack team adjusted the rabbitmq kernel configuration so that it can handle message passing between more than 1000 computing nodes in a region

physical machines, Xen VMS, KVM VMS and storage devices are managed and scheduled by tstack to improve server utilization and reduce it costs

what is the experience of the openstack community and other open source communities

Tencent actively participated in community activities, shared experiences and benefited a lot. At the global cloud computing open source conference in China, Tencent, as a participant and sponsor, conducted a series of technical demonstrations and participated in and passed the openstack interoperability test. Tencent also participated in China openstack day 2017 to share its product development and operation experience

tstack team is preparing openstack summit Sydney to share their past experience of using openstack. At the same time, they hope to learn from industry leaders from all over the world. In may2017, Tencent joined the CNCF and Linux foundations to make contributions to container services and KVM virtualization. In June, Tencent joined the MariaDB foundation to share the experience of database CDB

what to do next

openstack has brought many benefits to Tencent and paved the way for Tencent to continue to optimize openstack applications

in the future, Tencent will strengthen its support for openstack in the following aspects:

in addition to improving its ability to handle business growth, tsstack will also increase its investment in the community from this year. Tencent has top R & D personnel and developers in China, which will bring fresh blood to openstack community, provide high-quality code, share development and optimization experience, and hope to lead new projects. Tencent will provide its business optimization and internal modules to the community to help improve openstack. Tencent will also focus on magnum, electronic, kuryr and other projects, explore practical cases and share opinions. The team believes that this important new energy and knowledge sharing will enable openstack and the cloud computing industry to have a brighter future

lack of openstack operation skills is a common problem in enterprises. Strong technology requires strong operational skills. Therefore, Tencent will invest more in the construction of operation management platforms and tool chains to share experiences, lessons and best practices so that others can better use openstack to manage the cloud

Tencent will integrate various basic services, such as CMEM, CDB, CDN, message queue and load balancing, into tstack. Customers can access these basic services through API or GUI

Tencent will also work more closely with its partners to establish an open and integrated cloud computing ecosystem to reduce the cost of cloud users

Tencent is committed to the cloud computing market. Tsstack team will use openstack based private cloud tsstack and proprietary public cloud to establish a complete hybrid cloud service ecosystem for the global market

Tencent hopes to grow with openstack, make contributions to openstack and bring prosperity to openstack ecosystem

Copyright © 2011 JIN SHI