documents > User's Manual > Directory Quota

Directory Quota

English | Japanese

This document describes the directory quota of the Gfarm file system.

Overview

The directory quota limits file usage including number of inodes and total file size, under a set of directories called a dirset.
The same type of limitations can be enforced as the user and group quota.
It can work with the user and group quota. The strictest limit among them is applied.
Both creating a dirset and changing quota setting can be done by normal user privilege.
In the directory quota, gfarmroot group privilege is used for administration.

Limitation

If an ancestor directory already belongs to a dirset, you cannot make its descendant directories belong to a different dirset from the ancestor's dirset.
You cannot make a hardlink between different dirsets.
You can move a file between different dirsets, but you cannot move a directory between different dirsets, unless you have the gfarmroot group privilege.
Although gfarmroot group members can move a directory between different dirsets, it may make Gfarm metadata server's load high during its capacity recalculation if the number of files under the directory is a lot. Moreover, quota limit check cannot work precisely during the recalculation. The capacity before the movement is used instead of the real capacity until the recalculation is completed. Thus, such directory movement is only expected to be done during a maintenance period.
To add a directory to a dirset, the following conditions must be satisfied:
- The directory is empty.
- The user who executes the gfdirquota(1) command has to be the owner of the directory.
- The user who executes the gfdirquota(1) command has to be the creator of the dirset.
However, if the user has the gfarmroot group privilege, these conditions don't have to be satisfied. But the addition may make Gfarm metadata server's load high during capacity recalculation if the number of files under the directory is a lot. Moreover, quota limit check cannot work precisely during the recalculation. The capacity before the addition is used instead of the real capacity until the recalculation is completed. Thus, such directory addition is only expected to be done during a maintenance period.
There is no such feature to subdivide a directory quota further by using each user or each group. And there is no such feature to calculate amount of use of each user or group under the directory easily. You have to traverse entire directory to calculate such amount.
The only way to delete a directory from a dirset is to remove the directory by using gfrmdir (or rmdir under gfarm2fs).
To remove a dirset, you have to remove all directories in the dirset beforehand.

Example

Create two dirsets named dirset-a and dirset-b, and limit total capacity of dirset-a to 1GiB, and dirset-b to 2GiB.

$ gfdirquota -c dirset-a dirset-b
$ gfedquota -D dirset-a -h 1073741824
$ gfedquota -D dirset-b -h 2147483648

Add a directory /home/john-smith/dir-a to the dirset-a, and two directories named /home/john-smith/dir-b1 and /home/john-smith/dir-b2 to the dirset-b.

$ gfdirquota -a dirset-a /home/john-smith/dir-a
$ gfdirquota -a dirset-b /home/john-smith/dir-b1 /home/john-smith/dir-b2

Confirm the settings above.

$ gfdirquota
dirset-a
dirset-b
% gfdirquota -l
john-smith:dirset-a:
/home/john-smith/dir-a

john-smith:dirset-b:
/home/john-smith/dir-b1
/home/john-smith/dir-b2
$ gfquota -D dirset-a
UserName                 :             john-smith
DirsetName               :               dirset-a
GracePeriod              :               disabled
FileSpace                :                      0
FileSpaceGracePeriod     :               disabled
FileSpaceSoftLimit       :               disabled
FileSpaceHardLimit       :             1073741824
FileNum                  :                      0
FileNumGracePeriod       :               disabled
FileNumSoftLimit         :               disabled
FileNumHardLimit         :               disabled
PhysicalSpace            :                      0
PhysicalSpaceGracePeriod :               disabled
PhysicalSpaceSoftLimit   :               disabled
PhysicalSpaceHardLimit   :               disabled
PhysicalNum              :                      0
PhysicalNumGracePeriod   :               disabled
PhysicalNumSoftLimit     :               disabled
PhysicalNumHardLimit     :               disabled
$ gfquota -D dirset-b
UserName                 :             john-smith
DirsetName               :               dirset-b
GracePeriod              :               disabled
FileSpace                :                  49417
FileSpaceGracePeriod     :               disabled
FileSpaceSoftLimit       :               disabled
FileSpaceHardLimit       :             2147483648
FileNum                  :                     72
FileNumGracePeriod       :               disabled
FileNumSoftLimit         :               disabled
FileNumHardLimit         :               disabled
PhysicalSpace            :                  98834
PhysicalSpaceGracePeriod :               disabled
PhysicalSpaceSoftLimit   :               disabled
PhysicalSpaceHardLimit   :               disabled
PhysicalNum              :                    144
PhysicalNumGracePeriod   :               disabled
PhysicalNumSoftLimit     :               disabled
PhysicalNumHardLimit     :               disabled

You can use a directory name instead of a dirset name by using the -d option to see the quota limits and current amounts of use.
With -D option, only the owner of the dirset can see the quota limits.
With -d option, users who can write the directory can see them.

$ gfquota -d /home/john-smith/dir-b1
Pathname                 : gfarm://metadata-server:601/home/john-smith/dir-b1
UserName                 :             john-smith
DirsetName               :               dirset-b
GracePeriod              :               disabled
FileSpace                :                  49417
FileSpaceGracePeriod     :               disabled
FileSpaceSoftLimit       :               disabled
FileSpaceHardLimit       :             2147483648
FileNum                  :                     72
FileNumGracePeriod       :               disabled
FileNumSoftLimit         :               disabled
FileNumHardLimit         :               disabled
PhysicalSpace            :                  98834
PhysicalSpaceGracePeriod :               disabled
PhysicalSpaceSoftLimit   :               disabled
PhysicalSpaceHardLimit   :               disabled
PhysicalNum              :                    144
PhysicalNumGracePeriod   :               disabled
PhysicalNumSoftLimit     :               disabled
PhysicalNumHardLimit     :               disabled

If capacity recalculation is currently ongoing, and the current amounts of use is not accurate, the gfquota(1) command shows the following warning.

warning: quota usage is inaccurate

To remove a directory /home/john-smith/dir-b2 from the dirset-b, you have to remove the directory.

$ gfdirquota -l dirset-b
/home/john-smith/dir-b1
/home/john-smith/dir-b2
$ gfrmdir /home/john-smith/dir-b2
$ gfdirquota -l dirset-b
/home/john-smith/dir-b1

If you do not have the gfarmroot group privilege, directory movement between different dirsets fails as follows.

$ mkdir dir-a/dir
$ mv dir-a/dir dir-b1/
mv: cannot move `dir-a/dir' to `dir-b1/dir': Operation not supported

If you do not have the gfarmroot group privilege, adding non-empty directory to a dirset fails as follows.

$ gfmkdir -p /home/john-smith/dir-b3/dir
$ gfdirquota -a dirset-b /home/john-smith/dir-b3
gfarm://metadata-server:601/home/john-smith/dir-b3: directory not empty

directory_quota_rename_error_exdev option of gfarm2fs

If -o directory_quota_rename_error_exdev is specified as an option of the gfarm2fs(1) command, even normal user can move a directory between different dirsets by using the mv(1) command,

But the movement may be extremely slow, because the mv(1) command uses recursive file copy and recursive removal of the source directory tree to do this, and that means every file contents under the directory is copied from file system nodes to the client, and copied back from the client to file system nodes. And because of the behavior, some files and directories may not be moved, if you don't have enough access permission. Also, if there are files or directories which are not owned by you, the owner information will be lost.

Thus, it's not recommended to use the directory_quota_rename_error_exdev option in general. Rather, it's recommended to create same directory structure to the destination side, and use mv(1) command against files instead of the directory. Using mv(1) command against files is faster, because it only changes metadata, and no data copy happens.

Directory Quota

Overview

Limitation

Example

directory_quota_rename_error_exdev option of gfarm2fs

SEE ALSO