Maintaining Flash Disks
Replacing a Flash Disk Due to Flash Disk Failure
Each Exadata Storage Server is equipped withfourF20 PCIe cards. Each card has four flash disks (FDOMs) for a total of 16 flash disks. The four F20 PCIe cards are present in PCI slot numbers 1, 2, 4, and 5. The F20 PCIe cards are not hot-pluggable such that Exadata Storage Server must be powered down before replacing the flash disks or cards.
To identify a failed flash disk, use the following command:
CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk AND STATUS=critical DETAILname:[9:0:2:0]diskType:FlashDiskid:508002000092e70FMOD2luns:1_2makeModel:"MARVELL SD88SA02"physicalFirmware:D20RphysicalInsertTime:2009-10-27T13:11:16-07:00physicalInterface:sasphysicalSerial:508002000092e70FMOD2physicalSize:22.8880615234375GslotNumber:"PCI Slot: 1; FDOM: 2"status:critical
TheslotNumberattribute shows the PCI slot and the FDOM number.
If an flash disk is detected to have failed, then an alert is generated indicating that the flash disk, as well as the LUN on it, has failed. The alert message includes the PCI slot number of the flash card, and the exact FDOM number. These numbers uniquely identify the field replaceable unit (FRU). If you have configured the system for alert notification, then the alert will be sent by e-mail message to the designated address.
A flash disk outage can cause reduction in performance and dataredundancy. The failed disk should be replaced with a new flash disk at the earliest opportunity. If the flash disk is used for flash cache, then the effective cache size for the cell is reduced. If the flash disk is used for grid disks, then the Oracle ASM disks associated with these grid disks are automatically dropped with theFORCEoption from the Oracle ASM disk group and an Oracle ASM rebalance will ensue to restore the data redundancy.
The following procedure describes how to replace an FDOM due to disk failure
Inactivate all grid disks on the cell.
Shut down the cell.
Replace the failed flash disk based on the PCI number and FDOM number.
Power up the cell. The cell services are started automatically.
Bring all grid disks online using the following command:
CellCLI> ALTER GRIDDISK ALL ACTIVE
Verify that all grid disks have been successfully put online using the following command:
CellCLI> LIST GRIDDISK ATTRIBUTES asmmodestatus
Wait until asmmodestatus showsONLINEfor all grid disks.
The new flash disk is automatically used by the system. If the flash disk is used for flash cache, then the effective cache size will increase. If the flash disk is used for grid disks, then the grid disks will be re-created on the new flash disk. If those grid disks were part of an Oracle ASM disk group, then they will be added back to the disk group and the data will be rebalanced on them based on the disk group redundancy and ASM_POWER_LIMIT parameter.
Oracle ASM rebalance occurs when dropping or adding a disk. To check the status of the rebalance, do the following:
The rebalance operation may have been successfully run. Check the Oracle ASM alert logs to confirm.
The rebalance operation may be currently running. Check theGV$ASM_OPERATIONview to determine if the rebalance operation is still running.
The rebalance operation may have failed. Check theV$ASM_OPERATION.ERRORview to determine if the rebalance operation failed.
Rebalance operations from multiple disk groups can be done on different Oracle ASM instances in the same cluster if the physical disk being replaced contains ASM disks from multiple disk groups. One Oracle ASM instance can run one rebalance operation at a time. If all Oracle ASM instances are busy, then rebalance operations are queued.
Replacing a Flash Disk Due to Flash Disk Problems
Exadata Storage Server is equipped with fourF20 PCIe cards. Each card has four flash disks (FDOMs) for a total of 16 flash disks. The four F20 PCIe cards are present on PCI slot numbers 1, 2, 4, and 5. The F20 PCIe cards are not hot-pluggable such that Exadata Storage Server must be powered down before replacing the flash disks or cards.
You may need to replace a flash disk because the disk is inpredictive failurestatus orpoor performancestatus.
Toidentify apredictive failureflash disk, use the following command:
CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk AND STATUS=’predictive \failure’ DETAILname: [9:0:2:0]diskType: FlashDiskid: 508002000092e70FMOD2luns: 1_2makeModel: "MARVELL SD88SA02"physicalFirmware: D20RphysicalInsertTime: 2009-10-27T13:11:16-07:00physicalInterface: sasphysicalSerial: 508002000092e70FMOD2physicalSize: 22.8880615234375GslotNumber: "PCI Slot: 1; FDOM: 2"status: predictive failure坐在外婆的沙滩，看最白的帆影。
