000029682 - Failed OS Drive in Series 3 Supermicro appliance does not automatically rebuild

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000029682
Applies ToRSA Product Set: Security Analytics
RSA Product Set: Netwitness NextGen 
RSA Product/Service Type: SA Core Appliance
RSA Product/Service Type: NextGen Core Appliance
RSA Version/Condition: Security Analytics 10.0 and above. 
RSA Version/Condition: NextGen 9.8 and above
Platform: CentOS
O/S Version: 5, 6
Device: Series 3 Supermicro Appliance
IssueA new drive does not rebuild automatically after replacing the failed drive in an S3 Supermicro appliance following the steps outlined in How to re-attach the SuperMicro internal drive /dev/sdb on an RSA NetWitness appliance (KB article 26333). 
Note that sdh is the only drive listed in the raid. The second drive is missing indicated by the "U_" in the following example: 
[root@deco02 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdh1[0]
      270272 blocks super 1.0 [2/1] [U_]
     
md1 : active raid1 sdh2[0]
      155887488 blocks super 1.1 [2/1] [U_]
      bitmap: 1/2 pages [4KB], 65536KB chunk

You further confim an error when the new disk does not contain a valid partition table. 
[root@deco02 ~]# fdisk /dev/sdi
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x5b3ab281.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

 
ResolutionYou need to manually partition the new disk prior to adding the new disk into the RAID.
Determine the configuration of the first drive in the RAID mirror.  This example assumes the first, working drive is /dev/sdh and the new replacement drive is /dev/sdi.   Note the start and end cylinder numbers for each partition.  
All required commands and input in the examples below are identified with highlighted bold text.
[root@deco02 ~]# fdisk -l /dev/sdh
Disk /dev/sdh: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d6ffa
   Device Boot      Start         End      Blocks   Id  System
/dev/sdh1   *           1          34      270336   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdh2              34       19458   156018688   fd  Linux raid autodetect

Next partition the new drive using fdisk command to match the configuration of the existing, good drive. 
[root@deco02 ~]# fdisk /dev/sdi
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x5b3ab281.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19457, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-19457, default 19457): 34
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (35-19457, default 35): 35
Last cylinder, +cylinders or +size{K,M,G} (35-19457, default 19457): 19457
Command (m for help): p
Disk /dev/sdi: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5b3ab281
   Device Boot      Start         End      Blocks   Id  System
/dev/sdi1               1          34      273073+  83  Linux
/dev/sdi2              35       19457   156015247+  83  Linux
Command (m for help): a
Partition number (1-4): 1
Command (m for help): p
Disk /dev/sdi: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5b3ab281
   Device Boot      Start         End      Blocks   Id  System
/dev/sdi1   *           1          34      273073+  83  Linux
/dev/sdi2              35       19457   156015247+  83  Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Validate that /dev/sdi has now been correctly setup.
[root@ramkesdnetwit-deco02 ~]# fdisk -l /dev/sdi
Disk /dev/sdi: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5b3ab281
   Device Boot      Start         End      Blocks   Id  System
/dev/sdi1   *           1          34      273073+  83  Linux
/dev/sdi2              35       19457   156015247+  83  Linux

Now add the drive back into the raid and verify:
[root@deco02 ~]# mdadm --manage /dev/md0 --add /dev/sdi1
mdadm: added /dev/sdi1
[root@deco02 ~]# mdadm --manage /dev/md1 --add /dev/sdi2
mdadm: added /dev/sdi2
[root@deco02 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdi1[2] sdh1[0]
      270272 blocks super 1.0 [2/2] [UU]
     
md1 : active raid1 sdi2[2] sdh2[0]
      155887488 blocks super 1.1 [2/1] [U_]
      [>....................]  recovery =  0.1% (292864/155887488) finish=26.5min speed=97621K/sec
      bitmap: 1/2 pages [4KB], 65536KB chunk

 

Attachments

    Outcomes