OpenInfra Days North America 2024 at Indiana University

Exhibitors to include Platinum Sponsors: VEXXHOST and Rackspace; Gold Sponsor: Lightbits Labs; Reception Sponsors: IU Pervasive Technology Institute and Jetstream2

This session discusses the essentialls of monitoring an on-prem Cloud enviorment. It explains the indusrty standards in observability and monitoring, the best practices and the common pain points. It guides the audience on how to move from an ad-hoc monitoring/observability practices to a more standardized approach

The talk will also point to the differences of monitoring between different use-cases for the Cloud and where they intersect. The talk will end with a short demo of using Zabbix to monitor an on-prem Openstack environment.

Starting in 2017, the Minnesota Supercomputing Institute (MSI) built and operated a high-performance private cloud platform named "Stratus". As we embark on the design and build-out of a successor platform, this presentation covers some of the motivations and use cases which drove the development of the project, with a comprehensive analysis of what worked well and what fell short. We also discuss researcher experiences with Infrastructure as a Service (IaaS) capabilities, how these needs have changed over time, and how our next OpenStack cluster aims to address these challenges. By sharing our experiences at MSI, we aim to foster discussions on enhancing research computing ecosystems and better supporting researchers in this era of data-driven research.

We live in interesting times: new ideas and technological opportunities emerge at ever increasing rates in AI, edge computing and IoT, and programmable networking to name just a few. These innovations require a platform where they can be deployed and investigated -- and where new solutions -- that those disruptive ideas enable --  can be developed, tested, and shared. To support a breadth of experimentation such a platform has to provide access to a diversity of hardware, support deployments at scale, as well as deep reconfigurability so that a wide range of experiments can be supported. It also has to provide mechanisms for easy and direct sharing of repeatable digital artifacts so that new experiments and results can be easily replicated and help enable further innovation. Most importantly -- since science does not stand still – these types of instruments require the capability for constant adaptation to support an ever increasing range of experiments driven by emergent ideas and opportunities.

In this talk, I will describe the goals, the design strategy, and the capabilities of two NSF-funded scientific research platforms -- Chameleon and FLOTO – as well as some of the research and education projects our users are working on.

First, I will describe Chameleon which provides a large diversity of edge to cloud resources including Fugaku nodes, a range of accelerators, and disaggregated hardware (Liqid, GigaIO) with reconfigurability at bare metal level. Chameleon is currently distributed over two core sites at the University of Chicago and the Texas Advanced Computing Center (TACC) connected by 100 Gbps network. To date, the testbed has supported 10,000+ users and 1,100+ projects in research, education, and emergent applications.

FLOTO deploys 1,000 Raspberry PIs nationwide for broadband research. The FLOTO infrastructure implements “zero touch” installation for the devices such that they can be managed without physical access. Users can answer research different questions by deploying the devices in different configurations; running different applications (i.e., in this case, different types of broadband tests); or processing the data returned by the devices in different ways. Increasingly, users are also using FLOTO to answer different questions by customizing the infrastructure itself by e.g., combining it with new IoT sensors.

Exhibitors to include Platinum Sponsors: VEXXHOST and Rackspace; Gold Sponsor: Lightbits Labs; Reception Sponsors: IU Pervasive Technology Institute and Jetstream2

Join the members of the OpenInfra Foundations Diversity and Inclusion WG to learn about their efforts within the OpenInfra Projects and how you can get involved. We will also look for feedback on how to get more responses on the Foundation's Diversity Survey and other ways you can help grow the group and their efforts.

Our private cloud team at Workday has been using Zuul to build our own OpenStack distribution for close to four years. Up till now we’ve been using a hand-crafted VM hosting solution, but we decided to move to a Kubernetes hosted form of Zuul. We’ll briefly explain the self-hosting options we considered, the reasons why zuul-operator was selected, and what it requires. Then we’ll give you examples of how we use and operate it. This covers updating, configuration management, common tasks like holding jobs, and how to handle multiple tenants. Finally we’ll talk a bit about the internals of the operator and our adventures in getting CI up and running for it.

As cloud infrastructures grow in scale and complexity, the need for reliable, highly available distributed storage becomes more pressing. The OpenStack and Ceph communities have collaboratively developed an innovative solution: a highly available CephFS NFS service tailored specifically for OpenStack Manila. This novel approach ensures that OpenStack environments can benefit from the scalability and performance of CephFS, enabling the familiarity of Network File Systems (NFS) while maintaining the reliability and continuous availability that enterprise environments demand. In this talk, we will explore the technical underpinnings of this groundbreaking service, designed to meet the unique needs of OpenStack Manila consumers. Attendees will gain insights into the architecture, deployment strategies, and best practices for implementing this solution in their own OpenStack environments. We intend to dive into strategies employed to ensure high availability and resilience. and provide you with configuration best practices tailored to different use cases, from small-scale to large-scale enterprise deployments.

An update on the Ironic project status, direction, and then opportunity for participants to ask questions, and provide feedback to the project.

Kubernetes has become the de facto standard today for cloud-native development. The history of Kubernetes was always one of a stateless life. However, like most things, it is constantly evolving. Join this session and discover the journey for cloud-native success and the best way to innovate with multi-cloud, microservices, Kubernetes and agile methodologies as we move into the brave new world of Edge and the Cloud Native Continuum!

We will explore the process of migrating virtual machines (VMs) from VMware environments to OpenStack using open source tools. As organizations increasingly adopt cloud-native and open-source technologies, the need to seamlessly transition from traditional virtualization platforms like VMware to modern cloud infrastructures such as OpenStack has become crucial. We will discuss the challenges and strategies involved in this migration, including the selection of appropriate tools, preparation of the source and target environments, and the actual migration process. Key open source tools such as virt-v2v will be highlighted, demonstrating their roles in converting VM formats, adapting configurations, and ensuring compatibility. The session will cover best practices for minimizing downtime and ensuring data integrity during the migration, along with a live demonstration of the migration process. Attendees will gain practical insights into managing this transition efficiently and learn how to leverage open source solutions to achieve a successful VMware to OpenStack migration.

This talk will explore the challenges and solutions for companies grappling with the need for upgrades and more efficient operations within their OpenStack environments. We will focus on how businesses can navigate the complexities of migrating from legacy OpenStack setups to contemporary, agile infrastructures, ensuring that their infrastructure is not only up-to-date but also adaptable to future innovations.
Additionally, the presentation will examine migration strategies from other distributions to OpenStack, offering insights into how VEXXHOST’s Atmosphere can be seamlessly integrated into an organization's long-term cloud strategy to ensure scalability, flexibility, and security.

Orchestration engines, like Kubernetes, route network traffic between individual nodes over a virtual overlay network. The overlay network is opaque to existing network security and management tools. The ingress methodology is implemented strictly using the kubernetes API framework. It can’t be applied to all cases needed to integrate with external access points. Service Meshes standardize these needs within a central flexible methodology that relies more on “declarative" than on “imperative” application programming. Service Meshes also provide entry points for outside probes to be used for network management and telemetry gathering purposes under strict security control. Service Meshes can incorporate node labels within policies that can pin workloads to specific nodes, allowing for isolated deployments. There are currently many proxied Service Mesh options to choose from. The presentation will go through the various options and provide the pros and cons of each,

Cluster API (CAPI) is an open-source project that aims to simplify the provisioning and management of multiple Kubernetes clusters by providing declarative APIs and supporting tools. It has become a fundamental component within the systems of leading technology companies, including AWS, VMware, Azure, and Google.

However, it is currently limited in supporting automatic upgrades for the Kubernetes clusters it manages. Given the criticality of keeping Kubernetes clusters up-to-date with the latest features and improvements, and maintaining security, stability, performance, community support, and compatibility, this is a significant gap.

In this presentation, we will address the following topics:

1. Management of Kubernetes Clusters via Cluster API: We will explore the methodologies for managing Kubernetes clusters using Cluster API.

2. Integration of Automatic Upgrade Strategies: We will discuss integrating automatic upgrade strategies such as RollingUpgrade and BlueGreenUpgrade with CAPI for our customers. These strategies are crucial in ensuring high availability and stability and rollback solutions for Kubernetes clusters managed by Cluster API.

3. Management of Provider-Specific Component Upgrades: We will explain how to manage upgrades of provider-specific components and provide recommendations to users to minimize disruptions during the upgrade process.

Overview: We will explore the integration of Masakari, an OpenStack service for high availability of instances, with Consul, a widely adopted service mesh solution. This presentation will highlight the synergy between these two technologies, focusing on how they can be leveraged to enhance fault tolerance and service discovery in cloud environments.

Key Points:
1. Introduction to Masakari and Consul:
· Masakari: We'll start with an overview of Masakari, its role in ensuring high availability for virtual machine instances within OpenStack, and its core components such as instance monitoring, automated recovery, and notifications.
· Consul: We'll introduce Consul as a service mesh that provides service discovery, health checking, load balancing, and a key-value store. We'll discuss its popularity in managing microservices architectures.

2. Integration of Masakari with Consul:
· Brief Overview: We’ll briefly discuss why integrating Masakari with Consul is a powerful combination for managing highly available cloud environments.
· Detailed Explanation: We'll delve into the technical details of how Masakari can trigger Consul’s health checks and update service registries during failover events. We'll also cover how Consul can enhance Masakari’s failover process by providing real-time service discovery and more intelligent load balancing across recovered instances.

3. Use Cases and Benefits:
· Scenarios: We'll present real-world scenarios where combining these two technologies can significantly improve system resilience and reduce downtime.
· Benefits: Attendees will learn about the operational benefits, including improved fault tolerance, streamlined service discovery, and automated recovery processes.

4. Source Code and Contributions:
· Where is the code: Provide links and discuss about upstream source code for the project

· Recent Updates and Community Involvement: Give an overview of the latest updates and how we need more contributions from community members to expand the capability of Masakari including bug tracking.

Takeaways: By the end of this session, participants will have a clear understanding of how to integrate Masakari with Consul to build a more resilient cloud infrastructure. They will gain insights into the practical benefits of this integration and the technical steps required to mplement it in their environments. Attendees will walk away with actionable knowledge that they can apply to enhance the high availability and service management capabilities of their own cloud deployments.

Exhibitors to include Platinum Sponsors: VEXXHOST and Rackspace; Gold Sponsor: Lightbits Labs; Reception Sponsors: IU Pervasive Technology Institute and Jetstream2

In this talk, Joshua will share his insights and experiences with OpenTelemetry, an open-source project that offers protocols, APIs, and SDKs for collecting metrics, traces, and logs from applications and services. He will cover the comprehensive toolkit provided by the OpenTelemetry community, including language SDKs, the Collector, and the OTLP formats for metrics, traces, and logs. He will demonstrate how to instrument and monitor a microservices application running on a Kubernetes cluster, utilizing the full potential of OpenTelemetry. Attendees will learn how to use powerful open-source tools like Jaeger and Prometheus to effectively analyze telemetry signals from their applications. By the end of this session, attendees will have a solid understanding of how to implement OpenTelemetry in their projects, enhancing their debugging and observability practices. Join us as we delve into the world of OpenTelemetry, unlocking the capabilities of this powerful technology for your development needs.

Application timeouts can be a critical bottleneck in OpenStack environments, leading to frustrated end users, lost productivity, and blown SLAs. In this session, we'll illustrate how SDS transforms the performance and reliability of OpenStack environments by eliminating these timeouts. Discover how high-performance software-defined block storage engineered with intelligent data management and dynamic load balancing ensures your applications run smoothly, even for the most demanding workloads at scale.

Traditional network observability methods rely heavily on analytics and dashboards, requiring users to manually sift through extensive data to extract actionable insights. This approach proves challenging in large-scale clusters, where rapid decision-making is crucial. AI/ML offer a solution by automating the decision-making process, leveraging pre-existing human intelligence from historical and real-time data. This presentation examines the core aspects of network observability within virtual infrastructure deployments and explores the applicability of AI/ML on anomaly detection and troubleshooting, showcasing how AI/ML can enhance efficiency and responsiveness in network management.

A Forum session dedicated to OpenStack Kolla/Kolla-Ansible/Kayobe to discuss new features, feature requests

Exhibitors to include Platinum Sponsors: VEXXHOST and Rackspace; Gold Sponsor: Lightbits Labs; Reception Sponsors: IU Pervasive Technology Institute and Jetstream2

This talk explores how Mexico's regulatory framework is strategically designed to foster the adoption and growth of open source, driving digital transformation, democratization, and inclusion across the nation. We'll explore the concrete policies and initiatives that demonstrate the government's commitment to open source. From legislation mandating the use of open-source software in the public sector to the importance of institutions like Mexico’s National Research and Innovation Center for ICT (INFOTEC), dedicated to research, development, and implementation of open-source solutions in Mexico. Mexico is paving the way for a more open, accessible, and innovative digital ecosystem. Learn about real-world examples of open-source initiatives that are transforming various sectors. We believe that open technology has the potential to play a significant role in promoting our country’s digital transformation and digital inclusion in Mexico.

In the rapidly advancing technological landscape, transforming traditional data centers into sustainable green clouds is imperative. This session explores how AI and OpenStack are pivotal in this transformation. We will discuss how AI’s advanced analytics and predictive capabilities can optimize resource management, reduce energy consumption, and integrate renewable energy sources effectively. Concurrently, OpenStack’s open-source, modular architecture enables businesses to deploy scalable, efficient cloud infrastructure that supports sustainable practices. Key topics include: Leveraging AI for Enhanced Resource Management and Energy Efficiency: Discover how AI technologies can improve the management of IT resources and drive significant energy savings. Integrating Renewable Energy Sources: Learn how AI solutions can effectively incorporate renewable energy into cloud infrastructures. Utilizing OpenStack’s Modular Architecture: Understand how OpenStack’s flexible design helps build scalable and sustainable cloud environments. Benefits of Dynamic Resource Allocation and Reduced Carbon Footprints: Explore the advantages of adaptive resource management and its impact on reducing environmental footprints. Strategies for Combining AI and OpenStack: Gain insights into how the synergy between AI and OpenStack can create a greener, more resilient IT infrastructure. Real-World Examples: Review case studies of organizations successfully implementing AI and OpenStack to achieve sustainability goals.