LVM Limitation of Physical Partitions per Physical Volume


Contents

About this document
Overview
Disks larger than 4GB
Frequently asked questions
Changes introduced in AIX 4.3.1 and 4.3.2

About this document

This document discusses the limit of 1016 physical partitions per disk and how this limit affects the addition of disks larger than 4GB with the mkvg or extendvg commands. The first part of the document applies to AIX Versions 3.2, 4.1, 4.2, and 4.3.0. At the end of this document, there is a section that applies specifically to AIX Versions 4.3.1 and 4.3.2, discussing changes made to this limitation at AIX 4.3.1 and also some changes made at AIX 4.3.2.


Overview

In the design of Logical Volume Manager (LVM), each logical partition maps to one physical partition, and each physical partition maps to one or more physical partitions. Each physical partition maps to a number of disk sectors (only when mirroring, one LP maps to two or three physical partitions). The design of LVM limits the number of physical partitions that LVM can track per disk to 1016. In most cases, not all the possible 1016 tracking partitions are used by a disk. The default size of each physical partition during a mkvg command is 4MB, which implies that individual disks up to 4GB can be correctly tracked in a volume group.


Disks larger than 4GB

If a disk larger than 4GB is added to a volume group (based on usage of the default 4MB size for physical partition) the disk addition will fail with a warning message that the physical partition size needs to be increased (see Note 1 at the end of this section). There are two instances where this limitation will be enforced.

The first case is when the user tries to use mkvg to create a volume group in which the number of physical partitions on one of the disks in the volume group would exceed 1016. In this case, the user must pick from the available physical partition ranges, as follows:

   1, 2, (4), 8, 16, 32, 64, 128, or 256 MB 
The user must also employ the -s option with the mkvg.

The second case is when the user attempts to use extendvg to join the disk that violates the 1016 limitation with a pre-existing volume group. The user can either recreate the volume group with a larger physical partition size (which will allow the new disk to work with the 1016 limitation) or the user can create a standalone volume group (consisting of a larger physical partition size) for the new disk.

In AIX 4.2, 4.1 and 3.2.5, if the install code detects that the rootvg drive is larger than 4GB, it will change the mkvg -s value until the entire disk capacity can be mapped to the available 1016 tracks. (See Note 2 at the end of this section.) This install change also implies that all other disks added to rootvg, regardless of size, will also be defined at that new physical-partitions size.

NOTES:

  1. This bug was fixed in APAR IX48926, at AIX 3.2.5 and in bos.rte.lvm 4.1.3.0 at AIX 4.1. AIX Versions 3.2.5 and 4.1, which do not have this fix applied, will allow the creation of volume groups with more than 1016 partitions per disk. The implication of this bug allowing more than 1016 physical partitions is that the user may access all portions of the logical volume. However, during disk mirroring, the status of partitions beyond the 1016-limit will not be tracked correctly. If mirrors beyond the 1016 range become "stale", LVM will not be aware of their condition, and data consistency may become an issue for those partitions. Additionally, the migratepv and reorgvg commands create mirrors and delete them as a method for moving logical volumes within and between disks. If the 1016 limit is violated, then the migratepv or reorgvg commands may not behave correctly.
  2. This bug was fixed for AIX 3.2.5 rootvg install in APARs IX46862 and IX46863 and for AIX 4.1 with bos.rte.lvm 4.1.3.0.
  3. The 1016 PP/PV limitation is NOT enforced when importvg is run to define an existing volume group to the system; the limitation is only enforced when mkvg and extendvg are run. This means that a user can successfully define a volume group to the system that violates this limitation. However, any future extendvg operations on this volume group will fail for large disks, provided that the maintenance has been applied.

Frequently asked questions


Changes introduced in AIX 4.3.1 and 4.3.2

AIX 4.3.1 and later versions relax the limitation of 1016 physical partitions per physical volume, by introducing the concept of a volume group factor. When creating a volume group, you can specify your own PP/PV limitation, in multiples of 1016 (that is, 1016, 2032, 4064, and 8128). This is done via the -t flag on mkvg.

For example, the following command will create a volume group that will allow up to 2032 PPs per disk (with the default size of 4MB per PP):

    mkvg -y VGname -t 2 hdisk1 

Changes specific to AIX 4.3.2: In AIX 4.3.2 changes were made to the maximum number of disks a Volume Group can have. If the Volume Group was made in a big VG format (mkvg -B), or converted to a big VG format (chvg -B), you can add upto 128 disks to that Volume Group.

NOTES:

  1. The -B flag cannot be used if there are any stale physical partitions or there are any OPEN logical volumes in the volume group.
  2. Once the volume group is converted, it cannot be imported into AIX 4.3.1 or prior versions.
  3. The -B flag cannot be used if the volume group is varied on in concurrent mode.
  4. There must be enough free partitions available on each physical volume for the VGDA expansion for this operation to be successful.
  5. Since the VGDA resides on the edge of the disk and it requires contiguous space for expansion, the free partitions are required on the edge of the disk. If those partitions are allocated for user usage, they will be migrated to other free partitions on the same disk. The rest of the physical partitions will be renumbered to reflect the loss of the partitions for VGDA usage. This will change the mappings of the logical to physical partitions in all the PVs of this VG. If you have saved the mappings of the LVs for a potential recovery operation, you should generate the maps again after the completion of the conversion operation. Also, if the backup of the VG is taken with the map option and you plan to restore using those maps, the restore operation may fail since the partition number may no longer exist (due to reduction). It is recommended that you make a backup before the conversion, and that you also do so right after the conversion if the map option is utilized. Since the VGDA space has been increased substantially, every VGDA update operation (creating a Logical Volume, changing an Logical Volume, adding a PV, and so forth) may have a considerably longer duration.
  6. Once the Volume Group is converted to a big VG format, it cannot be brought back to to the regular format.

The disadvantage is that increasing the number of PPs allowed per disk will limit the number of disks allowed in the volume group. If the 1016 PP/PV limit is maintained, you can add up to 32 disks per volume group in AIX 4.3.1 and 128 disks in AIX 4.3.2. There is an inherent limit of 32512 PPs in the entire volume group in all levels of AIX EXCEPT version 4.3.2, where the limit is 130048 PPs.

For example: At AIX 4.3.1, if the PP/PV limit is changed to 4064 for a particular volume group, then only 8 drives may be placed in the volume group, even if those drives do not use 4064 PPs each. At AIX 4.3.2, if the Volume Group was created in a big Volume Group format, and the PP/PV limit is changed to 4064 for a particular volume group, then 32 disks may be placed in the Volume Group, since the maximum number of disks that can be a part of that Volume Group is now 128.

The chvg command also has a -t flag to allow you to change the PP/PV limit on an existing volume group, so that a larger drive could be added at that point. For example:

    chvg -t 2 VGname 

Only volume groups that have been created with a factor size other than 1, or that have been to a new factor size, can exceed the 1016 PP/PV limit.

Any volume group created outside the default factor size or changed to a different factor CANNOT be used on systems prior to AIX 4.3.1, even if the factor size is later changed back to 1. An example of the error received when accessing the volume group will look like the following:

    0516-002 lqueryvg: The volume group version is incompatible 
             with this level of the operating system and cannot 
             be activated. 

NOTE: These options are only available from the command line, not from SMIT.

At AIX version 4.3.1, the logical volumes in the Volume Group must be closed for chvg to successfully change the factor size. At AIX 4.3.2, the logical volumes need not be closed, so it is possible to change the factor size for the rootvg Volume Group also.




[ Doc Ref: 90605219814830     Publish Date: Oct. 16, 2000     4FAX Ref: 3689 ]