Alex Povolotsky (tarkhil) wrote in ru_freebsd,
Alex Povolotsky
tarkhil
ru_freebsd

После замены диска в zfs resilver встал

Добрый вечер

Сегодня, после броска питания (утверждается, что пропадание без пика), один из дисков в сервере вышел из строя (SMART утверждает, что хоть сейчас в космос, дисковые операции тормозят, в логе таймауты, smartctl -t long - срубание теста)

Диск заменен (не хотсвоп, с остановкой и выключением) 8 часов назад

На текущий момент
 # zpool status -v
  pool: pg_xlog
 state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Fri May 15 11:34:41 2015
config:

        NAME              STATE     READ WRITE CKSUM
        pg_xlog           ONLINE       0     0     0
          mirror-0        ONLINE       0     0     0
            gpt/pg_xlog0  ONLINE       0     0     0
            gpt/pg_xlog2  ONLINE       0     0     0

errors: No known data errors

  pool: system
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Tue Aug 25 15:50:43 2015
        2,15G scanned out of 271G at 73,1K/s, (scan is slow, no estimated time)
        2,14G resilvered, 0,79% done
config:

        NAME                       STATE     READ WRITE CKSUM
        system                     DEGRADED     0     0     0
          mirror-0                 DEGRADED     0     0     0
            gpt/system1            ONLINE       0     0     0
            replacing-1            OFFLINE      0     0     0
              2594766016928432125  OFFLINE      0     0     0  was /dev/gpt/system3/old
              gpt/system3          ONLINE       0     0     0  (resilvering)
        logs
          mirror-1                 ONLINE       0     0     0
            gpt/log0               ONLINE       0     0     0
            gpt/log2               ONLINE       0     0     0
        cache
          gpt/cache0               ONLINE       0     0     0
          gpt/cache2               ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 0h3m with 0 errors on Fri May 15 11:37:11 2015
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0p2  ONLINE       0     0     0
            ada2p2  ONLINE       0     0     0

errors: No known data errors

При этом 

 # zpool iostat -v
                              capacity     operations    bandwidth
pool                       alloc   free   read  write   read  write
-------------------------  -----  -----  -----  -----  -----  -----
pg_xlog                    2,07G  2,90G      0      7  6,75K  40,8K
  mirror                   2,07G  2,90G      0      7  6,75K  40,8K
    gpt/pg_xlog0               -      -      0      3  3,62K  41,0K
    gpt/pg_xlog2               -      -      0      3  3,47K  41,0K
-------------------------  -----  -----  -----  -----  -----  -----
system                      271G   657G     16     65  35,0K   216K
  mirror                    271G   657G     16     65  35,0K   191K
    gpt/system1                -      -     13     14   115K   145K
    replacing                  -      -      0     78      0   211K
      2594766016928432125      -      -      0      0      0      0
      gpt/system3              -      -      0     53      0   211K
logs                           -      -      -      -      -      -
  mirror                   1,07M  1,98G      0      0      0  25,0K
    gpt/log0                   -      -      0      0     49  25,0K
    gpt/log2                   -      -      0      0     49  25,0K
cache                          -      -      -      -      -      -
  gpt/cache0                598M  43,3G      0      0      4  19,6K
  gpt/cache2                584M  43,3G      0      0      4  19,1K
-------------------------  -----  -----  -----  -----  -----  -----
tank                       1,73G  3,24G      0     73  2,64K   222K
  mirror                   1,73G  3,24G      0     73  2,64K   222K
    ada0p2                     -      -      0     25  2,65K   223K
    ada2p2                     -      -      0     25  2,38K   223K
-------------------------  -----  -----  -----  -----  -----  -----

по iostat странный перекос нагрузки на ada2

Disks  ada0  ada1  ada2  ada3
KB/t   6,80  1,08  6,80  1,11 
tps      32    13    32    43 
MB/s   0,21  0,01  0,21  0,05
%busy     2     7    49   100 

и вообще никакая производительность у ada3

ada0/2 - SSD
ada1/3 - HDD



какие идеи? Что пробовать диагностировать?

FreeBSD 9.2-RELEASE-p13

real memory = 17179869184 (16384 MB)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
  • Post a new comment

    Error

    Comments allowed for members only

    Anonymous comments are disabled in this journal

    default userpic

    Your IP address will be recorded 

  • 4 comments