adm.cluster e storage.grid down

Description

07h25 e o Ganglia acusa a adm.cluster e a storage.grid down 05h15 min atras. O load da spg00 bateu em 318! O Nara acusa um pico de tr'afego das 05h ate as 06 de 30Mb/s. GFTP spraid,sRM-spraid e spraid_# offline para dcache.

[root@sprace:root]#ping spraid
PING spraid.if.usp.br (200.136.80.5) from 200.136.80.3 : 56(84) bytes of data.
64 bytes from spraid.if.usp.br (200.136.80.5): icmp_seq=1 ttl=64 time=1.09 ms
64 bytes from spraid.if.usp.br (200.136.80.5): icmp_seq=2 ttl=64 time=0.173 ms
--- spraid.if.usp.br ping statistics ---
2 packets transmitted, 2 received, 0% loss, time 1004ms
rtt min/avg/max/mdev = 0.173/0.634/1.095/0.461 ms
 
Mas nao responde ssh. Estamos UP para o resto.
 condor_status
na spgrid acusa todas as maquinas ok. A SPRace esta ok (estou digitando nela).

Updates

07h53 . A SPRaid voltou ao ar. Infelizmente me loguei no console mas foi inutil: a quantidade de logs

 Out of memory: process java ...
impossibilitaram qualquer operacao em qualquer terminal, entao rebootei na unha e o
fsck
deu conta do recado. O
ntpd
nao subiu OK Agora, no sprace
[root@spraid root]# /opt/d-cache/bin/dcache-core start
/pnfs/if.usp.br/ not mounted - going to mount it now ...
Starting dcache services:
Starting gridftp-spraidDomain  6 5 4 3 2 1 0 Done (pid=1133)
Starting srm-spraidDomain  6 5 4 3 2 1 0 Done (pid=1223)

[root@spraid root]# /opt/d-cache/bin/dcache-pool start

Starting dcache pool: Starting spraidDomain  6 5 4 3 2 1 0 Done (pid=1400)

[root@spraid root]# /etc/init.d/ntpd restart
Shutting down ntpd:                                        [  OK  ]
ntpd: Synchronizing with time server:                      [  OK  ]
Starting ntpd:                                             [  OK  ]
Demorou mas o monitoramento em http://spdc00.if.usp.br:2288/cellInfo acusou ok para os spraid_1, spraid_2, etc

Fulano em dd/mm/aaaa

Topic revision: r1 - 2006-10-06 - MarcoAndreFerreiraDias
 

This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback

antalya escort bursa escort eskisehir escort istanbul escort izmir escort