Kernel Config Variables   Virtualized /proc filesystem

SecureLinks

Symlink Owner Match Protection

Link Traversal Protection

ptrace Block

Xen XVDA detection

TPE Extension

IO Limits latency

Hybrid Kernel

Reading LVE usage

flashcache

OOM Killer for LVE Processes

File System Quotas

Starting from lvemanager 4.0-25.5 , lve-utils 3.0-21.2 , and cagefs-6.1-26 , CloudLinux OS utilities can read/write kernel config variables from a custom config /etc/sysctl.d/90-cloudlinux.conf (earlier, the parameters were read/written only from sysctl.conf ).

CloudLinux OS utilities get parameter by using sysctl system utility. So for now, even if a config variable is not set in the sysctl.conf and in the /etc/sysctl.d config files, this variable will be read by sysctl utility directly from /proc/sys .

If some kernel variable was set in /etc/sysctl.d/90-cloudlinux.conf do

sysctl --system

Starting from cagefs-6.1-27, fs.proc_can_see_other_uid will be migrated (one time) from /etc/sysctl.conf into /etc/sysctl.d/90-cloudlinux.conf . If this variable is not set in either file, it will default to 0. It is strongly advised against setting this variable in 90-cloudlinux.conf . Define it in /etc/sysctl.conf or in some other config file with an index number greater than 90-cloudlinux.conf , e.g. /etc/sysctl.d/95-custom.conf.

Starting from lve-utils-3.0-23.7 fs.proc_super_gid and fs.symlinkown_gid will be migrated (one time) from /etc/sysctl.conf into /etc/sysctl.d/90-cloudlinux.conf .

For lve-utils versions from 3.0-21.2 to 3.0-23.7 the migration was performed the same way, but during every package install/update. Variables setting guidelines are the same as for CageFS (see above).

You can prevent user from seeing processes of other users (via ps/top command) as well as special files in /proc file system by setting fs.proc_can_see_other_uid sysctl.

To do that, edit /etc/sysctl.conf

fs.proc_can_see_other_uid=0
fs.proc_super_gid=600

# sysctl -p

If fs.proc_can_see_other_uid is set to 0, users will not be able to see special files. If it is set to 1 - user will see other processes IDs in /proc filesystem.

fs.proc_super_gid=XX

The fs.proc_super_gid sets group ID which will see system files in /proc, add any users to that group so they will see all files in /proc. Usually needed by some monitoring users like nagios or zabbix and cldetect utility can configure few most commonly used monitoring software automatically.

Virtualized /proc filesystem will only display following files (as well as directories for PIDs for the user) to unprivileged users:

/proc/cpuinfo
/proc/version
/proc/stat
/proc/uptime
/proc/loadavg
/proc/filesystems
/proc/stat
/proc/cmdline
/proc/meminfo
/proc/mounts
/proc/tcp
/proc/tcp6
/proc/udp
/proc/udp6
/proc/assocs
/proc/raw
/proc/raw6
/proc/unix
/proc/dev

:::tip Note Starting from lve-utils 3.0-21.2, fs.proc_super_gid parameter in da_add_admin utility is written to /etc/sysctl.d/90-cloudlinux.conf. :::

In lve-utils-2.1-3.2 and later /proc can be remounted with hidepid=2 option to enable additional protection for procfs. This remount is performed in lve_namespaces service. This option is in sync with fs.proc_can_see_other_uid kernel parameter described above. When /etc/sysctl.conf does not contain fs.proc_can_see_other_uid setting, the protection is off (procfs is remounted with hidepid=0 option). In this case fs.proc_super_gid setting is ignored. Users are able to see full /proc including processes of other users on a server. This is a default behavior.

If /etc/sysctl.conf contains "fs.proc_can_see_other_uid=1" setting, then /proc will be remounted with "hidepid=0" option (disable “hidepid” protection for all users). If /etc/sysctl.conf contains "fs.proc_can_see_other_uid=0" setting, then /proc will be remounted with "hidepid=2" option (enable “hidepid” protection for all users). If /etc/sysctl.conf contains "fs.proc_can_see_other_uid=0" and "fs.proc_super_gid=$GID" settings, then /proc will be remounted with "hidepid=2, gid=$GID" options (enable “hidepid” for all users except users in group with gid $GID).

To apply /etc/sysctl.conf changes, you should execute

service lve_namespaces restart

/usr/share/cloudlinux/remount_proc.py

So, admin can prevent users from seeing processes of other users via "fs.proc_can_see_other_uid" and "fs.proc_super_gid" settings in /etc/sysctl.conf , like earlier.

Also, you can override this by specifying desired options for /proc in /etc/fstab .

To disable hidepid, add to /etc/fstab the following:

proc /proc proc defaults,hidepid=0,gid=0 0 0

proc /proc proc defaults,hidepid=2,gid=clsupergid 0 0

mount -o remount /proc

to apply /etc/fstab changes.
Nevertheless, we recommend to manage procfs mount options via /etc/sysctl.conf as described above for backward compatibility.

::: tip Note There is a known issue on CloudLinux 6 systems. User cannot see full /proc inside CageFS even when this user is in “super” group, that should see full /proc. This issue does not affect users with CageFS disabled. CloudLinux 7 is not affected. :::

::: tip Note Starting from lve-utils 3.0-21.2, lve_namespaces service can read parameters from the /etc/sysctl.d/90-cloudlinux.conf. :::

::: tip Note Even if fs.proc_can_see_other_uid and fs.proc_super_gid parameters are not set in config files but specified in /proc/sys, then when restarting lve_namespaces service the parameters from /proc/sys will be used. So, /proc will be remounted according to these parameters. :::

CloudLinux provides comprehensive protection against symbolic link attacks popular in shared hosting environment.

The protection requires setting multiple kernel options to be enabled.

fs.enforce_symlinksifowner

To protect against symlink attack where attacker tricks Apache web server to read some other user PHP config files, or other sensitive file, enable:

fs.enforce_symlinksifowner=1

Setting this option will deny any process running under gid fs.symlinkown_gid to follow the symlink if owner of the link doesn’t match the owner of the target file.

Defaults:

fs.enforce_symlinksifowner = 1
fs.symlinkown_gid = 48

When fs.enforce_symlinksifowner set to 1, processes with GID 48 will not be able to follow symlinks if they are owned by user1 , but point to file owned user2 .

Please, note that fs.enforce_symlinksifowner = 2 is deprecated and can cause issues for the system operation.

fs.symlinkown_gid

On standard RPM Apache installation, Apache is usually running under GID 48. On cPanel servers, Apache is running under user nobody, GID 99.

To change GID of processes that cannot follow symlink , edit the file /etc/sysctl.conf , add the line:

fs.symlinkown_gid = XX

$ sysctl -p

To disable symlink owner match protection feature, set fs.enforce_symlinksifowner = 0 in /etc/sysctl.conf , and execute

$ sysctl -p

::: danger /proc/sys/fs/global_root_enable [CloudLinux 7 kernel only] [applicable for kernels 3.10.0-427.36.1.lve1.4.42+] :::

proc/sys/fs/global_root_enable flag enables following the symlink with root ownership. If global_root_enable=0 , then Symlink Owner Match Protection does not verify the symlink owned by root.

For example, in the path /proc/self/fd , self is a symlink , which leads to a process directory.  The symlink owner is root . When global_root_enable=0 , Symlink Owner Match Protection excludes this element from the verification. When global_root_enable=1 , the verification will be performed, which could block the access to fd and cause violation of web site performance.

It is recommended to set /proc/sys/fs/global_root_enable=0 by default. If needed, set /proc/sys/fs/global_root_enable=1 to increase the level of protection.

::: tip Note Starting from lve-utils 3.0-21.2, fs.symlinkown_gid parameter values for httpd service user and fs.proc_super_gid for nagios service user is written to /etc/sysctl.d/90-cloudlinux.conf. :::

CageFS is extremely powerful at stopping most information disclosure attacks, where a hacker could read sensitive files like /etc/passwd .

Yet, CageFS does not work in each and every situation. For example, on cPanel servers, it is not enabled in WebDAV server, cPanel file manager and webmail, as well as some FTP servers don’t include proper change rooting.

This allows an attacker to create symlink or hardlink to a sensitive file like /etc/passwd and then use WebDAV , filemanager, or webmail to read the content of that file.

Starting with CL6 kernel 2.6.32-604.16.2.lve1.3.45, you can prevent such attacks by preventing user from creating symlinks and hardlinks to files that they don’t own.

This is done by set following kernel options to 1:

fs.protected_symlinks_create = 1
fs.protected_hardlinks_create = 1

::: danger We do not recommend to use protected_symlinks option for cPanel users as it might break some of the cPanel functionality. :::

::: tip Note Link Traversal Protection is disabled by default for the new CloudLinux OS installations/convertations. :::

fs.protected_symlinks_create = 0
fs.protected_hardlinks_create = 0

fs.protected_symlinks_allow_gid = id_of_group_linksafe
fs.protected_hardlinks_allow_gid = id_of_group_linksafe

To manually adjust the settings, edit: /etc/sysctl.d/cloudlinux-linksafe.conf and execute:

sysctl -p /etc/sysctl.d/cloudlinux-linksafe.conf

sysctl --system

:::tip Note Starting from lvemanager 4.0-25.5, if there is no /etc/sysctl.d/cloudlinux-linksafe.conf config file, selectorctl for PHP with --setup-without-cagefs and --revert-to-cagefs keys writes fs.protected_symlinks_create and fs.protected_hardlinks_create parameters to /etc/sysctl.d/90-cloudlinux.conf. :::

Starting with kernel 3.10.0-427.18.s2.lve1.4.21 ( CloudLinux 7) and 2.6.32-673.26.1.lve1.4.17 ( CloudLinux 6) we re-implemented ptrace block to protect against ptrace family of vulnerabilities. It prevents end user from using any ptrace related functionality, including such commands as strace, lsof or gdb .

By default, CloudLinux doesn't prevent ptrace functionality.

Defaults:

kernel.user_ptrace = 1
kernel.user_ptrace_self = 1

The option kernel.user_ptrace disables PTRACE_ATTACH functionality, option kernel.user_ptrace_self disables PTRACE_TRACEME .

To disable all ptrace functionality change both sysctl options to 0, add this section to /etc/sysctl.conf :

## CL. Disable ptrace for users
kernel.user_ptrace = 0
kernel.user_ptrace_self = 0
##

Apply changes with:

$ sysctl -p

Different software could need different access to ptrace , you may need to change only one option to 0 to make them working. In this case, there will be only partial ptrace protection.

::: danger ptrace protection is known to break PSA service for Plesk 11 :::

2.6.32 kernels have different mode of naming Xen XVDA drives.

By adding xen_blkfront.sda_is_xvda=0 to kernel boot line in grub.conf you will make sure no naming translation is done, and the drives will be identified as xvde .

By default, this option is set to 1 in the kernel, and drives are detected as xvda . This is needed only for CloudLinux 6 and Hybrid kernels.

TPE Extension will removed in the next version of CloudLinux 5.x kernel

CloudLinux 5.x (kernel 2.6.18) has limited support for trusted path execution extension. CloudLinux 6.x (kernel 2.6.32) and CloudLinux 5.x with hybrid kernel don't have TPE extension

TPE (Trusted Path Execution)

The kernel supports TPE feature out of the box. You can configure it using following files:

·        /proc/sys/kernel/grsecurity/grsec_lock
·        /proc/sys/kernel/grsecurity/tpe
·        /proc/sys/kernel/grsecurity/tpe_gid
·        /proc/sys/kernel/grsecurity/tpe_restrict_all

To enable TPE feature in a standard way just add following to the end of your /etc/sysctl.conf

#GRsecurity 
kernel.grsecurity.tpe = 1 
kernel.grsecurity.tpe_restrict_all = 1 
kernel.grsecurity.grsec_lock = 1  

And do:

# sysctl -p

::: tip Note Once you set grsec_lock to 1, you will not be able to change TPE options without reboot. :::

This Trusted Path Execution feature was adopted from grsecurity.

[lve1.2.29+]

When customer reaches IO Limit, the processes that are waiting for IO will be placed to sleep to make sure they don't go over the limit. That could make some processes sleep for a very long time. By defining IO latency, you can make sure that no process sleeps due to IO limit for more then X milliseconds. By doing so, you will also let customers to burst through the limits, and use up more than they were limited too in some instances.

This option is OFF by default.

For CloudLinux 6 and CloudLinux 7 (since Hybrid kernel lve1.4.x.el5h):

To enable IO Limits latency and set it to 10 seconds:

# echo 10000 > /sys/module/kmodlve/parameters/latency

# echo 2000000000 > /sys/module/kmodlve/parameters/latency

It is possible to set, for example, 1000 as a permanent value. To do so, create a file /etc/modprobe.d/kmodlve.conf with the following content:
options kmodlve latency=1000

For CloudLinux 5 (OBSOLETE):

To enable IO Limits latency and set it to 10 seconds:

# echo 10000 > /sys/module/iolimits/**parameters/latency

# echo 2000000000 > /sys/module/iolimits/**parameters/latency

CloudLinux 6 Hybrid kernel

CloudLinux 6 Hybrid Kernel is CloudLinux 7 (3.10.0) kernel compiled for CloudLinux 6 OS. New 3.10 kernel features a set of performance and scalability improvements related to IO , networking and memory management, available in CloudLinux 7 OS . It also features improved CPU scheduler for better overall system throughput and latency.

Please find information on the main features of 3.10 kernel branch on the links:

https://kernelnewbies.org/Linux_3.10#head-e740f930dfd021616cc42e8abf21c79d0b07e217

https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.10.1

CloudLinux 7 Hybrid kernel

CloudLinux 7 Hybrid Kernel is essentially EL8-based (4.18) kernel compiled for CloudLinux OS 7.

You can find more information on 4.18 kernel branch using this link:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/8.0_release_notes/new-features#kernel

How to migrate from the normal to hybrid channel (CL6h):

::: tip Note The system must be registered in CLN. :::

1. Update rhn-client-tools from production

2. Run normal-to-hybrid script.

3. Reboot after script execution is completed.

yum update rhn-client-tools
normal-to-hybrid
reboot

How to migrate from the normal to hybrid channel (CL7h):

::: tip Note The system must be registered in CLN. :::

1. Update rhn-client-tools rhn-check rhn-setup from testing repository

2. Run normal-to-hybrid script.

3. Reboot after script execution is completed.

yum update rhn-client-tools rhn-check rhn-setup --enablerepo=cloudlinux-updates-testing
normal-to-hybrid
reboot

How to migrate from hybrid to the normal channel (for both CL6h and CL7h):

::: tip Note The system should be registered in CLN. :::

1. Run hybrid-to-normal script.

2. Reboot after script execution is completed.

hybrid-to-normal
reboot

Known limitations and issues of CloudLinux 6 Hybrid kernel :

1. We do not remove Hybrid kernel after migration from Hybrid to the normal channel, but we remove linux-firmware package which is needed to boot Hybrid kernel. This is because CloudLinux 6 does not allow to remove the package of currently running kernel. Proper removal procedure will be implemented, but for now, we should warn users not to boot Hybrid kernel if they have migrated to normal channel .

2. Kernel module signature isn't checking for now, as 3.10 kernel is using x509 certificates to generate keys and CL6 cannot detect signatures created in such way. The solution will be implemented.

Known limitations and issues of CloudLinux 7 Hybrid kernel

Features that are absent in the current kernel build:

1. CPU boost
2. CRIU support for mod_lsapi PRO
3. Per LVE traffic accounting

Limitations of the current kernel build:

1. Standard OOM killer is used
2. The /etc/sysctl.conf parameter proc_can_see_other_uid is supported but its behavior is more restrictive than documented. This will be improved in the next beta release.
3. Symlink Owner Match Protection is disabled by default. To enable it, use sysctl utility:

sysctl -w fs.enforce_symlinksifowner=1

Find more details on symlink owner match protection

CloudLinux kernel provides real time usage data in file.

All the statistics can be read from that file in real time. Depending on your kernel version you will get either Version 6 of the file, or version 4 of the file. You can detect the version by reading the first line of the file. It should look like:

6:LVE... for version 6
4:LVE... for version 4

First line presents headers for the data. Second line shows default limits for the server, with all other values being 0. The rest of the lines present limits & usage data on per LVE bases.

Version 6 (CL6 & hybrid kernels):

6:LVE        EP        lCPU        lIO        CPU        MEM        IO        lMEM        lEP        nCPU        fMEM        fEP        lMEMPHY        lCPUW        lNPROC        MEMPHY        fMEMPHY        NPROC        fNPROC
0        0        25        1024        0        0        0        262144        20        1        0        0        262144        100        0        0        0        00
300        0        25        1024        1862407        0        0        262144        20        1        0        0        262144        100        0        31        000

* Available only for x86_64, CloudLinux 6 and Hybrid servers

Flashcache is a module originally written and released by Facebook (Mohan Srinivasan, Paul Saab and Vadim Tkachenko ) in April of 2010. It is a kernel module that allows Writethrough caching of a drive on another drive. This is most often used for caching a rotational drive on a smaller solid-state drive for performance reasons. This gives you the speed of an SSD and the size of a standard rotational drive for recently cached files. Facebook originally wrote the module to speed up database I/O , but it is easily extended to any I/O .

To install on CloudLinux 6 & Hybrid servers:

$ yum install flashcache

More info on flashcache : https://github.com/facebook/flashcache/

ArchLinux has a good page explaining how to use flashcache : https://wiki.archlinux.org/index.php/Flashcache

When LVE reaches its memory limit, the processes inside that LVE are killed by OOM Killer and appropriate message is written to /var/log/messages . When any LVE hits huge number of memory limits in short period of time, then OOM Killer could cause system overload. Starting from kernel 2.6.32-673.26.1.lve1.4.15 ( CloudLinux 6) and from kernel 3.10.0-427.18.2.lve1.4.14 ( CloudLinux 7) heavy OOM Killer could be disabled. If so - lightweight SIGKILL will be used instead.

By default OOM Killer is enabled, to disable it please run:

For CloudLinux 6 :

# echo 1 > /proc/sys/ubc/ubc_oom_disable

Also, add the following to /etc/sysctl.conf file to apply the same during boot:

ubc.ubc_oom_disable=1

For CloudLinux 7:

# echo 1 > /proc/sys/kernel/memcg_oom_disable

Also, add the following to /etc/sysctl.conf file to apply the same during boot:

kernel.memcg_oom_disable=1

In Ext4 file system, the process with enabled capability CAP_SYS_RESOURCE is not checked on the quota exceeding by default. It allows userland utilities selectorctl and cagefs to operate without fails even if a user exceeds a quota.

To disable quota checking in XFS file system set cap_res_quota_disable option to 1 using the following command:

# echo 1 > /proc/sys/fs/xfs/cap_res_quota_disable