000027944 - How to check  local file system disk space usage for RSA Identity Governance & Lifecycle

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support on Jun 7, 2019
Version 4Show Document
  • View in full screen mode

Article Content

Article Number000027944
Applies ToRSA Product Set: Identity Governance & Lifecycle
Platform: Appliance
IssueThere are times when the local file system uses up all available space and new files cannot be created.  Various errors can occur as a result. A typical one is the following Operating System error:

Linux Error: 28: No space left on device


Behavior can include:



  • The JBoss/WildFly server not starting or responding,
  • The Oracle instance not starting or responding (due to log files not being able to be written),
  • Tasks not completing, such as collections, backups, etc.

Hardware appliances have two file systems:



Software appliances have just one local file system where all files, including the physical database files, are located. A software appliance does not use the +ASM partition.
TasksThis article addresses the following tasks:
  1. Determine if this is a hardware or software appliance. This tells you whether or not there might be physical Oracle database files to consider.
  2. Review the existing file system and identify which files could be moved or deleted to free up disk space.
Resolution

How to determine space usage



To clear the Linux Error: 28, you must first identify which files or directories are consuming the most space. This can be done using the UNIX commands: df, du and find.  Refer to the man pages for complete details on these commands.



The df command can be used to see how much space is available on the devices mounted on your system. We recommend using the parameters -kh, for the output to be more easily understood. The output is similar to that shown below. In these examples, there is not a disk space issue. Systems that have problems would typically show 95% or higher used.



  • On Red Hat:


$ df -kh
Filesystem                       Size   Used   Avail  Use%   Mounted on
/dev/mapper/VolGroup00-LogVol00   51G    13G    36G    27%     /
/dev/sda1                         99M    13M    82M    13%     /boot
tmpfs                             2.0G     0   2.0G     0%     /dev/shm


  • On SUSE:


~> df -kh
Filesystem                        Size   Used   Avail   Use%   Mounted on
/dev/mapper/system-root            58G    8.2G    47G    16%      /
udev                              2.0G    100K    2.0G    1%      /dev
/dev/sda1                          61M     30M     29M   51%      /boot


To be able to see the actual space used by files and directories, the UNIX command du should be used. The output of this command depends on the current directory location, and if a directory parameter is used. If there are a large number of subdirectories, the redirect option can be used to redirect the output to a file, and then the file reviewed for details. The examples shown below are meant as guidelines for the use of this command:




$ cd /tmp
$ du -h
8.0K ./.ICE-unix
160K ./aveksa/postinstall
168K ./aveksa
12K ./.font-unix
12K ./.X11-unix
228K .

$ du -hs
228K .

$ du -h aveksa
160K aveksa/postinstall
168K aveksa

$ du -hs aveksa
168K aveksa

$ du -h aveksa > aveksa_size.txt

$ more aveksa_size.txt
160K aveksa/postinstall
168K aveksa


Another command that can be used is a combination of the find command with the -type and -size parameters. The -size option allows you to look for files over a certain size. This value can be adjusted as needed as you look for files which can be candidates for removal. In this example, the . option is used in the find command.  The command as shown means to search everything in the directory under the current directory for files larger than 20MB. The directory location could be changed to /, to indicate search everything under the root folder. Note that regardless of the search location, permission error messages may occur, depending on the current user account. It may be useful to switch to the 'root' account in this situation.



In this example, the command used is find . -type f -size +20000k -exec du -h {} \;




$ pwd
/tmp
$ find . -type f -size +20000k -exec du -h {} \;
72M ./aveksa/packages/jdk1.6.0_45_x64.tar.bz2
1.1G ./aveksa/packages/p10404530_112030_Linux-x86-64_2of7.zip
93M ./aveksa/packages/jboss-4.2.2.GA.zip
935M ./aveksa/packages/p10404530_112030_Linux-x86-64_3of7.zip
22M ./aveksa/packages/instantclient-basiclite-linux-x86-64-11.2.0.2.0.zip
1.3G ./aveksa/packages/p10404530_112030_Linux-x86-64_1of7.zip
find: ./aveksa/staging/database/lib: Permission denied
find: ./aveksa/staging/database/validation: Permission denied
find: ./aveksa/staging/database/reportdefs: Permission denied
find: ./aveksa/staging/database/Tests: Permission denied
find: ./aveksa/staging/database/custom: Permission denied
find: ./aveksa/staging/database/SampleData: Permission denied
find: ./aveksa/staging/database/DBA: Permission denied
25M ./aveksa/staging/test-verification-plugins.zip
52M ./aveksa/staging/AFX-2.8.1-Standard-Connectors.zip
719M ./aveksa/staging/ACM-WebSphere-6.8.1.tar
720M ./aveksa/staging/ACM-WebLogic-6.8.1.tar
718M ./aveksa/staging/aveksa.ear



Common locations for files eligible for removal



Do not delete any files unless appropriate backups are created first.  That noted, the removal of any of the files listed below will not keep the RSA Identity Governance & Lifecycle  application from working.



This is not a complete list of files that can be removed, but is a start on what files can be looked at for consideration for removal.



1.  WildFly/JBoss and Application Server logs



  • The aveksaServer.log is rolled based on file size, not date created. There may be an aveksaServer.log.1, .2, .3, .4 and .5.  These logs may be deleted once they are reviewed and it is determined that they do not contain relevant or necessary diagnostic information.   

2.  Database  log and trace files



  • Both the Oracle AVDB and +ASM instance (on appliances and some software installations) create .trc files by default, even when there are no problems. These .trc files may be deleted. An example of how to identify these .trc files using a Linux command is shown here:


$ cd /u01
$ find . -name *.trc  -exec ls -l {} \;


  • Both the AVDB and +ASM alert logs are not rolled automatically. They are appended to until they are manually deleted or renamed. The Oracle instance must be stopped to be able to manually rename and/or purge these files.  They should be reviewed for errors or pertinent diagnostic information before removal. These files are typically found at:
    • /u01//app/oracle/diag/rdbms/avdb/AVDB/trace/alert_AVDB.log
    • /u01/app/oracle/diag/asm/+asm/+ASM/trace/alert_+ASM.log

3.  Database backup files



The Oracle backup .dmp files (created when you export (backup) the database) may not be (located in the /home/aveksa/AveksaExportImportDir directory) may not be gzipped (compressed), which would consume more space than when they are gzipped. The number of .dmp files may accumulate if they are not periodically moved off the system. The impact of several .dmp files will vary depending on how often they are created and their size.



4.  Application installation files



The directory /tmp/aveksa/packages can be removed, as long as these installation files are backed up, and as long as it is understood that if the application needs to be re-installed, they might need to be restored from a backup or downloaded again from RSALink.  It is not recommended to delete the /tmp/aveksa/staging directory, unless absolutely necessary and the same caveat as with the packages directory applies.



5.  Potential Operating System system core dump files:



These files are a dump of memory and can be quite large. They should be moved off the system for future analysis to determine why a core file was created. They are typically named core or core.pid (where pid is the process ID of the process which created the core file).  They can be found by searching for core* from the / partition as the root user.



6.  Application java profiling files



The RSA Identity Governance and Lifecycle application has the capability to be started with Java profiling enabled, for diagnostic purposes. Java profile snapshots can then be created. These files can be quite large and should be moved off the system as soon as possible, and profiling disabled when no longer required. After moving these files, they can then be deleted.  These profiles are created with a datestamp in the name in the /home/oracle/snapshots directory, and can be found with a command similar to this:




$ ls -l /home/oracle/Snapshots/JBoss*.snapshot


7.  Archived redo logs



Although we do not support the use of RMAN, archiving of redo log files might be enabled, and if there is no remote archive location set up, the files may end up in this default location with a similar name:




/u01/app/oracle/product/12.1.0/db_1/dbs/arch1_499_918293958.dbf

In the database, check for the following:


sqlplus / as sysdba
SQL> ARCHIVE LOG LIST

Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            /u01/app/oracle/product/12.1.0/db_1/dbs
Oldest online log sequence     11160
Next log sequence to archive   11163
Current log sequence           11163


Check if archiving is enabled, and if the files go to the local file system.

Attachments

    Outcomes