Recently, while working, I came across many OpenStack nova code reviews, so I decided to write a few articles to document how nova works. The first article will start with the nova-scheduler component. It will guide readers through the code level to understand how the conductor calls the scheduler, and how the scheduler determines the selection process.

This article uses the latest stable version, Victoria, as its foundation.

nova-scheduler overview

nova-scheduler is designed as a plugin-based scheduler, allowing users to customize their own scheduler drivers based on their specific requirements. By default, nova uses the filter scheduler provided within its own codebase. Filter Scheduler。

The working principle of the Filter Scheduler is quite straightforward. It passes all hosts through a series of "filters." Hosts that do not meet the filter conditions are removed, while those that pass move on to the final selection pool. These surviving hosts then undergo weighing to determine priority, and finally, one host is randomly selected from the top few to host the new VM. A simple flowchart can be found in the official documentation:

OpenStack provides some built-inCommon Filters, and users can also create and use their own custom filters based on their needs.Weigher There are also some commonly used ones, and users can develop their own as needed.

What methods does the nova-scheduler driver have?

We can explore Nova-Scheduler Driver Base Class found four methods in:

• __init__
• run periodic tasks
• hosts_up
• select destinations

__init__

The init method's name clearly indicates initialization of this class, where you obtain the host manager and service group API as two object instances.

run periodic tasks

This method, like others, is also quite easy to understand from the code. Some of its behaviors, such as running services or topics, are directly passed through without any implementation in the base class.

hosts_up

hosts_up will return a list containing hosts that are running services or topics.

select destinations

select_destinations is the core of the entire scheduler; the logic for selecting hosts is implemented in this method. nova-conductor calls this method when scheduling an instance.

How does the nova-scheduler work overall?

Currently, most internal tasks in nova are handled by the conductor. From the conductor manager's code, we can find a responsible component. schedule instance function：

def _schedule_instances(self, context, request_spec,
                        instance_uuids=None, return_alternates=False):
    scheduler_utils.setup_instance_group(context, request_spec)
    with timeutils.StopWatch() as timer:
        host_lists = self.query_client.select_destinations(
            context, request_spec, instance_uuids, return_objects=True,
            return_alternates=return_alternates)
    LOG.debug('Took %0.2f seconds to select destinations for %s '
              'instance(s).', timer.elapsed(), len(instance_uuids))
    return host_lists

From the code, we can see that the conductor manager starts here scheduler's query client to call select_destinations This method ultimately uses RPC to send the request to the nova-scheduler driver’s select destination for processing.

After applying the filters and weighers mentioned above, select destination will return a list containing each instance's destination object. Selection Object。

Summary

The overall flow of OpenStack nova-scheduler is actually quite simple and easy to understand. The driver-based design also allows users to easily replace the scheduler with their own custom requirements, even if using the built-in filter scheduler, they can easily customize or extend their own filters. This design is considered quite reasonable, though currently, when there are many hosts and instances to schedule simultaneously, performance issues may arise, possibly requiring splitting nova cells to address them.

Reference

https://docs.openstack.org/nova/latest/user/filter-scheduler.html
https://github.com/openstack/nova/tree/stable/victoria

---

Copyright Notice: All articles in this blog are licensed under CC BY-NC-SA 4.0 unless otherwise stated.