Disclaimer
The information here represents my personal findings using published documents from Oracle. It doesn't represent my legal opinion. I am not a lawyer. Take this information and fight for your right as a consumer/customer and demand an official response from Oracle by email, not verbal.Introduction
I've had numerous encounters with customers citing Oracle sales people stating that virtualizing Oracle DB on VMware is not supported, and that the licensing of the entire physical host's cores, or even the entire cluster's cores is mandatory, and in a nut shell: this is NOT entirely true and can be circumvented.The information below is based on Oracle's legal documents and licensing documents and guidelines. Check the references for the links and details.
References are denoted with numbers. When you see #1 it means see reference number 1 at the end of this post.
Executive Summary (TL;DR)
The Oracle partitioning guide is not a contractual document and Oracle strictly states it's for educational purposes only. Therefore it cannot use it to impose how customers should partition their environments or systems. |
| Excerpt from Oracle's "Oracle Partitioning Policy" document |
Details and Resources
The only contractually obligating documents from Oracle are:- Technical Support Policy document
- Processor Core Factor table
- Oracle License and Service Agreement (OLSA) / Oracle Master Agreement(OMA)
Terminology and Concepts
Alright, let's get into details, and one step at a time to provide a full picture. First things first:License Types
Oracle DB is licensed in different ways, depending on its edition:- By number of users using the connected application or whose data are saved in the DB
- By number of CPU sockets. A socket is a full physical processor, regardless of the number of cores inside it.
- By number of CPU cores. This is the case for the Enterprise edition of the DB. Cores refer to the number of physical cores in every CPU socket installed in the server. Threads are considered logical cores, and you do not license those; only the physical cores.
Audit Compliance
Before moving forward, let's discuss audit compliance. Oracle audit team can request to audit your environment. You have the choice of not allowing them, but they might come back later with government officials to enforce it, or if you open a support ticket, they'd do an inspection anyway to see whether you're eligible for support or not.If/When you do allow Oracle's audit team to run an audit, it's essential to agree on a scope and limited time to do the activity: i.e., clusters 1, 2 and 3 and the activity is to not exceed 2 weeks maximum. This is to be stated in the legal document you'll be signing prior to them starting the audit activity.
Oracle will ask you extract virtual machine (VM) activity logs, where they've been and where they've moved for X number for months. It's also important to limit the period: 1-3 months should be sufficient for any audit activity.
If you do not impose such limits, Oracle can keep asking for extended periods to run their scripts, and try to find at least 1 case of deviation to impose penalties on you.
Some sales people will scare you that you're violating Oracle terms and will be subject to penalties of millions of USD.
As long as you're complying with the rules below, and have done the settings properly and have full log of all VM activity (in vCenter) to prove that the VMs haven't moved beyond the licensed hosts and cores, no one can penalize you.
If they continue to harass you, ask them to send you an official email, and once you receive it, send it to Oracle's USA legal team. You'll receive a note from them acknowledging that you're in full compliance.
Feel free to reach out to me and I'll help you reach the right people within Oracle. For planning and designing help with your setup to make sure you're compliant prior to deploying the Oracle workloads, I can offer this within Kuwait only. If you're outside of Kuwait or the Arabian Gulf region, I suggest you contact a vendor selling x86, another selling IBM POWER and let both give you 5-year Total Cost of Ownership (TCO) studies including Oracle software license costing.
License Core Factor
Oracle applies different ratios of licenses needed for each core depending on the processor/CPU being used in the servers [#1]. This is called Core Ratio, and usually for Intel mid-range processors (Intel E5-2400, E5-2600, Xeon Silver and Xeon Gold), the core factor is 0.5.For Oracle's own SPARC CPU, the core facor for M5, M6, M7 and M8 CPUs is 0.5. This is in bid to push for its own hardware and provide a full solution.
For higher-end processors (Intel E7-4800, E7-8800, Xeon Platinum, and IBM POWER), the core factor is 1.0.
Processor Choices and License Calculation
If your application vendor says they need 10 cores, you have to ask them to specify which processor and model have they benchmarked their database workload on.It's unfortunate that many software vendors benchmark their workload once (say 2012 for example), and then keep using the same hardware requirements on newer systems, which means customers (you) end up with an extremely over-sized solution.
Why? Because 10 Intel Xeon E5 cores in 2012 are equal to about 6 Intel Xeon Gold cores now (rough estimate). The same applies to any processor brand, such as IBM POWER and Oracle/Sun/Fujitsu SPARC. The enhancements vary, but the idea is the same: do not believe the software vendor's requirements unless they tell you which hardware was used to do the benchmark.
If the application vendor says you need 10 cores on Intel Xeon Gold, then you need to purchase 10 (cores) x 0.5 (Xeon Gold core factor) = 5 Oracle Enterprise DB core licenses.
If the application vendor says you need 10 cores on IBM POWER9, then you need to purchase 10 (cores) x 1.0 (IBM P9 core factor) = 10 Oracle Enterprise DB core licenses.
The above does NOT mean that POWER core factor is more expensive than Intel, because the performance of 10 cores on Intel is less than 10 cores on POWER. That's why Oracle assigned POWER processors a higher core factor. However, I will NOT discuss which processor to choose in this post, to not derail from the topic of licensing.
The advice I give all my clients is: Choose the platform that gives you the best Return on Investment (most cost effective) and reliability. Make sure to always factor in cost of software and hardware for 5 years for your solutions, including maintenance, support and subscription costs.
Virtualization/Partitioning Types
Virtualization allows you to simultaneously run multiple virtual machines (VMs), each with its own operating system (OS), on the same physical server.Oracle treats hypervisors (virtualization engines/software) differently, mainly as a sales tactic (politics) and not for technical differentiating factors (though some technical factors exist, but the main drive remains to push sales into their direction).
Oracle's list of supported virtualization and partitioning technologies, VMware's vSphere is not listed, for political reasons, but it's fully technically functional and support is provided as follows:
- If the problem is already known, Oracle will provide support.
- If the problem is unknown, Oracle require you to reproduce the issue on a physical server.
- Some of my customers said they've had tickets open on supported platforms for months, while Oracle support engineers threw the blame on Microsoft Windows, and Windows support engineers threw the blame on Oracle. So you can imagine how this might turn on an unsupported platform.
- VMware openly states that it will provide full support for Oracle software running on VMware's platform, so you contact VMware and they'll use their in-house Oracle certified support engineers. See the links in references for details on what VMware covers. [#4]
- Oracle sales people might tell you you'll never get support, but that's a lie. Ask them to email you their claim, and then you can escalate that. 99.99% they won't dare email you since it's illegal to make such claims.
Licensing types based on virtualization: Oracle licensing states that you need to license every CPU core that's used by the database. That's easy to do on Unix platforms, but requires additional configurations on x86 (AMD/Intel) platforms.
x86 (AMD/Intel) systems
As x86 systems are considered commodity servers, they don't offer a function to isolate specific cores for specific workloads/VMs. However, with VMware vSphere or Microsoft Hyper-V hypervisors, you can assign specific processor cores to always be used by a specific VM. Hyper-V calls it CPU Pinning. vSphere calls it CPU Affinity.
Keep in mind, you need to also restrict which hosts are able to run these virtual machines, in addition to the CPU core affinity. On VMware vSphere, when enabling High Availability, a VM will restart on a different host if the original host lost power, therefore you need to set cluster policies to have the VMs run on specific hosts only, even in cases of host failures.
Remember the audit section above? This is why you need to setup such restrictions here.
Example: You have a VMware cluster of 4 hosts, each host has 2x 14-core Intel processors (28 total). You have/need Oracle Enterprise Edition DB effective licenses for 8 cores = 16 Intel cores licensed (0.5 core factor license for Intel mid-range CPUs).
You want to run 2 instances of Oracle DB as virtual machines, each with 8 cores (4 effective core licenses).
You can easily create a Host Affinity rule in VMware's Distributed Resource Scheduler to restrict the DB VMs to specific 2 hosts in the cluster, and edit the VM settings to specify 8 cores in each host as part of the CPU Affinity settings. This way you lock the 2 VMs to 2 specific hosts in the cluster, and each VM to specific CPU cores.
There is no need to buy dedicated servers for Oracle with the CPU cores matching the license. You do need to license any host cores that will run Oracle instances (2 hosts to have high availability -- if one VM goes offline, the other is still functional. Do not power on the other one.
If you need to do maintenance on one physical server: power off one of the 2 VMs, and carry on your maintenance, then power it on when the host is ready.
Remember the audit section above? This is why you need to setup such restrictions here.
Example: You have a VMware cluster of 4 hosts, each host has 2x 14-core Intel processors (28 total). You have/need Oracle Enterprise Edition DB effective licenses for 8 cores = 16 Intel cores licensed (0.5 core factor license for Intel mid-range CPUs).
You want to run 2 instances of Oracle DB as virtual machines, each with 8 cores (4 effective core licenses).
You can easily create a Host Affinity rule in VMware's Distributed Resource Scheduler to restrict the DB VMs to specific 2 hosts in the cluster, and edit the VM settings to specify 8 cores in each host as part of the CPU Affinity settings. This way you lock the 2 VMs to 2 specific hosts in the cluster, and each VM to specific CPU cores.
There is no need to buy dedicated servers for Oracle with the CPU cores matching the license. You do need to license any host cores that will run Oracle instances (2 hosts to have high availability -- if one VM goes offline, the other is still functional. Do not power on the other one.
If you need to do maintenance on one physical server: power off one of the 2 VMs, and carry on your maintenance, then power it on when the host is ready.
Unix (POWER/SPARC) systems
Such systems allow cores to be pooled/grouped and specific workloads can be restricted to certain cores. This is known as Hard Partitioning.
I am not very familiar with SPARC systems, so my example(s) will be for POWER: if you have a machine with 2 CPUs, 10 cores each, for a total of 20 cores, you can create a Shared Processor Pool of 6 cores and restrict all Oracle DB VMs/LPARs to run on that pool only. This allows you to license only 6 cores, and the VMs will share those 6 cores.
It's often that customers buy dedicated core licenses for each DB they create, however, in many times when we did utilization analysis of those VMs and DBs, the CPU utilization was much lower than the assigned values, however the customer had a huge number of total cores licensed for Oracle DB!
A better approach is to create a pool for the DBs, and let the VMs use the cores from that pool. Additionally, on POWER, it's possible to allow a VM to have 2 cores, but increase number of cores if needed, then scale back. In such a scenario, it will never exceed the restriction impose by the pool, so you always remain within the license boundries.
I am not very familiar with SPARC systems, so my example(s) will be for POWER: if you have a machine with 2 CPUs, 10 cores each, for a total of 20 cores, you can create a Shared Processor Pool of 6 cores and restrict all Oracle DB VMs/LPARs to run on that pool only. This allows you to license only 6 cores, and the VMs will share those 6 cores.
It's often that customers buy dedicated core licenses for each DB they create, however, in many times when we did utilization analysis of those VMs and DBs, the CPU utilization was much lower than the assigned values, however the customer had a huge number of total cores licensed for Oracle DB!
A better approach is to create a pool for the DBs, and let the VMs use the cores from that pool. Additionally, on POWER, it's possible to allow a VM to have 2 cores, but increase number of cores if needed, then scale back. In such a scenario, it will never exceed the restriction impose by the pool, so you always remain within the license boundries.
The above setup helps with one part of the audit, but when it comes to moving VMs/LPARs around different physical hosts, the same rules apply: you have to license the physical cores where the VMs run. So, if you have 2 physical hosts for High Availability, there are few ways to do the setup:
License Options
Licensing One VM only
Oracle licenses per installed instance. If you create a VM with an OS and install Oracle DB on it, you have to license it, even if it's offline/powered off.To license one VM only in HA setup, you have to use storage replication, or connect both hosts to the same storage, such that at any time, only one VM instance exists on the servers. When you need to failover to your 2nd site or host, you do the job manually and import/power on the VM on the 2nd host, as long as it's powered off and removed from the 1st host.
Additionally, you have to disable Live Partition Mobility, vMotion or any function that allows VMs to move between hosts in the same cluster.
This is true for both x86 and Unix.
Licensing Multiple VMs
If you wish to use Oracle DataGuard or Real Active Cluster (RAC) to guarantee data consistency using application/DB-level replication, then you need to license at least 2 instances and setup the replication scheme on the DB level.Additionally, you have to disable Live Partition Mobility, vMotion or any function that allows VMs to move between hosts in the same cluster. You license cores on physical hosts that have the VMs running. If you do want the VMs to move to other hosts, you'll need to license all hosts permissible for movement.
This is true for both x86 and Unix.
As you can see, these legal restrictions are not technical limitations, but only to enforce customers to pay more licenses and/or enforce Oracle's own ecosystem onto the customer to further leverage more purchases in the future.
Pitfalls
As you can see above, when using x86 systems, there's some added overhead on the operations team to make sure the VMs always remain in compliance when doing daily operations and maintenance jobs. It's easier to do things when having Unix systems and maintain compliance, but then you need operations people with Unix skills.A mistake of 1 person in operations could put you out of compliance. If you run a small company, you may be better off with buying dedicated physical servers for Oracle workloads (but end up with many physical boxes). If you're an enterprise with many Oracle workloads, I suggest moving away from them if possible, and if not, go with a Unix environment that gives you flexibility and is able to reduce your overall cost on software licenses.
