Ubuntu 11.04 server regular crashes

Post Reply
gritto
Posts: 3
Joined: Tue Dec 27, 2011 10:03 pm

Ubuntu 11.04 server regular crashes

Post by gritto »

Hello,
we have a similar problem as http://www.fit-pc.com/forum/viewtopic.php?f=58&t=2375 except we are in the Ubuntu 11 version.
we are experiencing a serious Hd crashes on Samsung 500GB sata disk on FitPc2 with 2GHz dual core processor.
The machine is running 2 ext4 partitions:
- the first 30GB is managing the root and installation disk
- the second partition is handling the storage of Zone Minder video recorder images.
after a random number of days we have the machine crashed and we have to reboot it. in 2 cases we cannot restore it and we had to reclone the machine .
we did 20 machines and only 10 are working now.
attached is an image of the last crash.
ssh could be provided
regards
Andrea
Attachments
27122011070-1.jpg
27122011070-1.jpg (136.75 KiB) Viewed 13047 times

gritto
Posts: 3
Joined: Tue Dec 27, 2011 10:03 pm

Re: Ubuntu 11.04 server regular crashes

Post by gritto »

here are a more detailed dmesg log:

Code: Select all

[70466.263939] Buffer I/O error on device sda5, logical block 38764
[70466.263945] Buffer I/O error on device sda5, logical block 38765
[70466.263951] Buffer I/O error on device sda5, logical block 38766
[70466.263958] Buffer I/O error on device sda5, logical block 38767
[70466.263964] Buffer I/O error on device sda5, logical block 38768
[70466.263970] Buffer I/O error on device sda5, logical block 38769
[70466.263977] Buffer I/O error on device sda5, logical block 38770
[70466.263983] Buffer I/O error on device sda5, logical block 38771
[70466.263990] Buffer I/O error on device sda5, logical block 38772
[70466.263996] Buffer I/O error on device sda5, logical block 38773
[70466.264021] Buffer I/O error on device sda5, logical block 38774
[70466.264027] Buffer I/O error on device sda5, logical block 38775
[70466.264034] Buffer I/O error on device sda5, logical block 38776
[70466.264040] Buffer I/O error on device sda5, logical block 38777
[70466.264046] Buffer I/O error on device sda5, logical block 38778
[70466.264052] Buffer I/O error on device sda5, logical block 38779
[70466.264059] Buffer I/O error on device sda5, logical block 38780
[70466.264065] Buffer I/O error on device sda5, logical block 38781
[70466.264071] Buffer I/O error on device sda5, logical block 38782
[70466.264077] Buffer I/O error on device sda5, logical block 38783
[70466.264086] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 45 (offset 12288 size 262144 starting block 7363200)
[70466.264130] sd 0:0:0:0: [sda] Unhandled error code
[70466.264135] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.264143] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 03 82 d4 00 00 03 70 00
[70466.264158] end_request: I/O error, dev sda, sector 58905600
[70466.264165] Buffer I/O error on device sda5, logical block 38784
[70466.264172] Buffer I/O error on device sda5, logical block 38785
[70466.264178] Buffer I/O error on device sda5, logical block 38786
[70466.264184] Buffer I/O error on device sda5, logical block 38787
[70466.264190] Buffer I/O error on device sda5, logical block 38788
[70466.264197] Buffer I/O error on device sda5, logical block 38789
[70466.264203] Buffer I/O error on device sda5, logical block 38790
[70466.264209] Buffer I/O error on device sda5, logical block 38791
[70466.264215] Buffer I/O error on device sda5, logical block 38792
[70466.264222] Buffer I/O error on device sda5, logical block 38793
[70466.264228] Buffer I/O error on device sda5, logical block 38794
[70466.264234] Buffer I/O error on device sda5, logical block 38795
[70466.264240] Buffer I/O error on device sda5, logical block 38796
[70466.264246] Buffer I/O error on device sda5, logical block 38797
[70466.264253] Buffer I/O error on device sda5, logical block 38798
[70466.264259] Buffer I/O error on device sda5, logical block 38799
[70466.264265] Buffer I/O error on device sda5, logical block 38800
[70466.264271] Buffer I/O error on device sda5, logical block 38801
[70466.264277] Buffer I/O error on device sda5, logical block 38802
[70466.264284] Buffer I/O error on device sda5, logical block 38803
[70466.264290] Buffer I/O error on device sda5, logical block 38804
[70466.264296] Buffer I/O error on device sda5, logical block 38805
[70466.264302] Buffer I/O error on device sda5, logical block 38806
[70466.264308] Buffer I/O error on device sda5, logical block 38807
[70466.264315] Buffer I/O error on device sda5, logical block 38808
[70466.264321] Buffer I/O error on device sda5, logical block 38809
[70466.264327] Buffer I/O error on device sda5, logical block 38810
[70466.264333] Buffer I/O error on device sda5, logical block 38811
[70466.264340] Buffer I/O error on device sda5, logical block 38812
[70466.264346] Buffer I/O error on device sda5, logical block 38813
[70466.264352] Buffer I/O error on device sda5, logical block 38814
[70466.264358] Buffer I/O error on device sda5, logical block 38815
[70466.264364] Buffer I/O error on device sda5, logical block 38816
[70466.264370] Buffer I/O error on device sda5, logical block 38817
[70466.264377] Buffer I/O error on device sda5, logical block 38818
[70466.264383] Buffer I/O error on device sda5, logical block 38819
[70466.264389] Buffer I/O error on device sda5, logical block 38820
[70466.264395] Buffer I/O error on device sda5, logical block 38821
[70466.264403] Buffer I/O error on device sda5, logical block 38822
[70466.264409] Buffer I/O error on device sda5, logical block 38823
[70466.264416] Buffer I/O error on device sda5, logical block 38824
[70466.264422] Buffer I/O error on device sda5, logical block 38825
[70466.264429] Buffer I/O error on device sda5, logical block 38826
[70466.264437] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 46 (offset 98304 size 176128 starting block 7363243)
[70466.264446] Buffer I/O error on device sda5, logical block 38827
[70466.264455] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 47 (offset 0 size 4096 starting block 7363244)
[70466.264464] Buffer I/O error on device sda5, logical block 38828
[70466.264470] Buffer I/O error on device sda5, logical block 38829
[70466.264477] Buffer I/O error on device sda5, logical block 38830
[70466.264483] Buffer I/O error on device sda5, logical block 38831
[70466.264492] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 47 (offset 4096 size 16384 starting block 7363248)
[70466.264500] Buffer I/O error on device sda5, logical block 38832
[70466.264507] Buffer I/O error on device sda5, logical block 38833
[70466.264513] Buffer I/O error on device sda5, logical block 38834
[70466.264520] Buffer I/O error on device sda5, logical block 38835
[70466.264526] Buffer I/O error on device sda5, logical block 38836
[70466.264532] Buffer I/O error on device sda5, logical block 38837
[70466.264538] Buffer I/O error on device sda5, logical block 38838
[70466.264545] Buffer I/O error on device sda5, logical block 38839
[70466.264551] Buffer I/O error on device sda5, logical block 38840
[70466.264557] Buffer I/O error on device sda5, logical block 38841
[70466.264563] Buffer I/O error on device sda5, logical block 38842
[70466.264569] Buffer I/O error on device sda5, logical block 38843
[70466.264576] Buffer I/O error on device sda5, logical block 38844
[70466.264582] Buffer I/O error on device sda5, logical block 38845
[70466.264588] Buffer I/O error on device sda5, logical block 38846
[70466.264594] Buffer I/O error on device sda5, logical block 38847
[70466.264604] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 47 (offset 20480 size 65536 starting block 7363264)
[70466.264613] Buffer I/O error on device sda5, logical block 38848
[70466.264619] Buffer I/O error on device sda5, logical block 38849
[70466.264625] Buffer I/O error on device sda5, logical block 38850
[70466.264631] Buffer I/O error on device sda5, logical block 38851
[70466.264638] Buffer I/O error on device sda5, logical block 38852
[70466.264644] Buffer I/O error on device sda5, logical block 38853
[70466.264650] Buffer I/O error on device sda5, logical block 38854
[70466.264656] Buffer I/O error on device sda5, logical block 38855
[70466.264663] Buffer I/O error on device sda5, logical block 38856
[70466.264669] Buffer I/O error on device sda5, logical block 38857
[70466.264675] Buffer I/O error on device sda5, logical block 38858
[70466.264681] Buffer I/O error on device sda5, logical block 38859
[70466.264687] Buffer I/O error on device sda5, logical block 38860
[70466.264696] Buffer I/O error on device sda5, logical block 38861
[70466.264703] Buffer I/O error on device sda5, logical block 38862
[70466.264710] Buffer I/O error on device sda5, logical block 38863
[70466.264717] Buffer I/O error on device sda5, logical block 38864
[70466.264725] Buffer I/O error on device sda5, logical block 38865
[70466.264733] Buffer I/O error on device sda5, logical block 38866
[70466.264740] Buffer I/O error on device sda5, logical block 38867
[70466.264747] Buffer I/O error on device sda5, logical block 38868
[70466.264754] Buffer I/O error on device sda5, logical block 38869
[70466.264762] Buffer I/O error on device sda5, logical block 38870
[70466.264768] Buffer I/O error on device sda5, logical block 38871
[70466.264775] Buffer I/O error on device sda5, logical block 38872
[70466.264782] Buffer I/O error on device sda5, logical block 38873
[70466.264790] Buffer I/O error on device sda5, logical block 38874
[70466.264801] Buffer I/O error on device sda5, logical block 38875
[70466.264808] Buffer I/O error on device sda5, logical block 38876
[70466.264815] Buffer I/O error on device sda5, logical block 38877
[70466.264822] Buffer I/O error on device sda5, logical block 38878
[70466.264828] Buffer I/O error on device sda5, logical block 38879
[70466.264835] Buffer I/O error on device sda5, logical block 38880
[70466.264842] Buffer I/O error on device sda5, logical block 38881
[70466.264849] Buffer I/O error on device sda5, logical block 38882
[70466.264855] Buffer I/O error on device sda5, logical block 38883
[70466.264861] Buffer I/O error on device sda5, logical block 38884
[70466.264867] Buffer I/O error on device sda5, logical block 38885
[70466.264873] Buffer I/O error on device sda5, logical block 38886
[70466.264880] Buffer I/O error on device sda5, logical block 38887
[70466.264886] Buffer I/O error on device sda5, logical block 38888
[70466.264892] Buffer I/O error on device sda5, logical block 38889
[70466.264898] Buffer I/O error on device sda5, logical block 38890
[70466.264904] Buffer I/O error on device sda5, logical block 38891
[70466.264910] Buffer I/O error on device sda5, logical block 38892
[70466.264917] Buffer I/O error on device sda5, logical block 38893
[70466.264925] EXT4-fs warning (device sda5): ext4_end_bio:242: I/O error writing to inode 47 (offset 86016 size 188416 starting block 7363310)
[70466.264957] sd 0:0:0:0: [sda] Unhandled error code
[70466.264962] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.264973] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 01 87 59 b8 00 00 48 00
[70466.264989] end_request: I/O error, dev sda, sector 25647544
[70466.914440] JBD2: Detected IO errors while flushing file data on sda1-8
[70466.914483] Aborting journal on device sda1-8.
[70466.915190] EXT4-fs error (device sda1) in ext4_da_writepages:3033: IO failure
[70466.915218] sd 0:0:0:0: [sda] Unhandled error code
[70466.915224] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.915233] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 01 84 08 00 00 00 08 00
[70466.915250] end_request: I/O error, dev sda, sector 25430016
[70466.915257] quiet_error: 15 callbacks suppressed
[70466.915264] Buffer I/O error on device sda1, logical block 3178496
[70466.915269] lost page write due to I/O error on sda1
[70466.916307] sd 0:0:0:0: [sda] Unhandled error code
[70466.916315] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.916324] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 13 c2 18 00 00 00 08 00
[70466.916346] end_request: I/O error, dev sda, sector 331487232
[70466.916356] Buffer I/O error on device sda5, logical block 34111488
[70466.916361] lost page write due to I/O error on sda5
[70466.916475] JBD2: I/O error detected when updating journal superblock for sda5-8.
[70466.916557] JBD2: Detected IO errors while flushing file data on sda5-8
[70466.916572] JBD2: I/O error detected when updating journal superblock for sda1-8.
[70466.920074] EXT4-fs error (device sda1) in ext4_reserve_inode_write:5641: Journal has aborted
[70466.920327] sd 0:0:0:0: [sda] Unhandled error code
[70466.920333] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.920342] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 00 08 00 00 00 08 00
[70466.920358] end_request: I/O error, dev sda, sector 2048
[70466.920367] Buffer I/O error on device sda1, logical block 0
[70466.920372] lost page write due to I/O error on sda1
[70466.920418] EXT4-fs (sda1): Remounting filesystem read-only
[70466.920852] sd 0:0:0:0: [sda] Unhandled error code
[70466.920859] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.920867] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 74 17 c8 00 00 08 00
[70466.920883] end_request: I/O error, dev sda, sector 7608264
[70466.920891] Buffer I/O error on device sda1, logical block 950777
[70466.920896] lost page write due to I/O error on sda1
[70466.920960] sd 0:0:0:0: [sda] Unhandled error code
[70466.920966] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.920973] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 54 83 10 00 00 08 00
[70466.920989] end_request: I/O error, dev sda, sector 5538576
[70466.920996] Buffer I/O error on device sda1, logical block 692066
[70466.921001] lost page write due to I/O error on sda1
[70466.921112] sd 0:0:0:0: [sda] Unhandled error code
[70466.921118] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.921125] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 74 27 c0 00 00 08 00
[70466.921140] end_request: I/O error, dev sda, sector 7612352
[70466.921146] Buffer I/O error on device sda1, logical block 951288
[70466.921151] lost page write due to I/O error on sda1
[70466.921249] sd 0:0:0:0: [sda] Unhandled error code
[70466.921256] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.921268] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 74 af 90 00 00 08 00
[70466.921284] end_request: I/O error, dev sda, sector 7647120
[70466.921296] Buffer I/O error on device sda1, logical block 955634
[70466.921302] lost page write due to I/O error on sda1
[70466.921475] sd 0:0:0:0: [sda] Unhandled error code
[70466.921485] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.921492] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 03 05 4d 50 00 00 08 00
[70466.921507] end_request: I/O error, dev sda, sector 50679120
[70466.921513] Buffer I/O error on device sda1, logical block 6334634
[70466.921518] lost page write due to I/O error on sda1
[70466.964525] sd 0:0:0:0: [sda] Unhandled error code
[70466.964536] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.964546] EXT4-fs (sda1): previous I/O error to superblock detected
[70466.967712] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 60 d0 60 00 00 08 00
[70466.967741] end_request: I/O error, dev sda, sector 6344800
[70466.970955] Buffer I/O error on device sda1, logical block 792844
[70466.974322] EXT4-fs warning (device sda1): ext4_end_bio:242: I/O error writing to inode 1570347 (offset 1466368 size 4096 starting block 793101)
[70466.974399] sd 0:0:0:0: [sda] Unhandled error code
[70466.974407] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.974416] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 00 08 00 00 00 08 00
[70466.974439] end_request: I/O error, dev sda, sector 2048
[70466.977805] Buffer I/O error on device sda1, logical block 0
[70466.980942] lost page write due to I/O error on sda1
[70466.980987] EXT4-fs (sda1): ext4_da_writepages: jbd2_start: 353 pages, ino 1570347; err -30
[70466.984120] JBD2: Detected IO errors while flushing file data on sda1-8
[70466.987384] sd 0:0:0:0: [sda] Unhandled error code
[70466.987395] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70466.987407] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 03 7e 18 00 00 00 08 00
[70466.987432] end_request: I/O error, dev sda, sector 58595328
[70466.990636] Buffer I/O error on device sda5, logical block 0
[70466.993885] lost page write due to I/O error on sda5
[70466.993917] EXT4-fs error (device sda5): ext4_journal_start_sb:296: Detected aborted journal
[70467.000514] EXT4-fs (sda5): Remounting filesystem read-only
[70467.003892] EXT4-fs (sda5): previous I/O error to superblock detected
[70467.008663] sd 0:0:0:0: [sda] Unhandled error code
[70467.008675] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70467.008687] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 03 7e 18 00 00 00 08 00
[70467.008713] end_request: I/O error, dev sda, sector 58595328
[70467.012567] EXT4-fs (sda5): ext4_da_writepages: jbd2_start: 1024 pages, ino 48; err -30
[70470.965148] init: mysql main process (899) terminated with status 1
[70470.965245] init: mysql main process ended, respawning
[70471.091687] init: mysql main process (3702) terminated with status 1
[70471.091776] init: mysql main process ended, respawning
[70471.124666] sd 0:0:0:0: [sda] Unhandled error code
[70471.124676] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70471.124686] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 02 04 1f e0 00 00 08 00
[70471.124709] end_request: I/O error, dev sda, sector 33824736
[70471.128548] sd 0:0:0:0: [sda] Unhandled error code
[70471.128558] sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[70471.128568] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 02 04 1f e0 00 00 08 00
[70471.128591] end_request: I/O error, dev sda, sector 33824736
We had other 3 machine with this problem.
we think that temperature cuold damage the disk, but it is not possible that after some time the disk carashes in this way.

any ideas?
Regards
Andrea

gabrielh
Site Admin
Posts: 1260
Joined: Thu Jun 02, 2011 1:13 pm

Re: Ubuntu 11.04 server regular crashes

Post by gabrielh »

This can be caused by storage disks overheating, what type of the disks are you using?
Please connect the disks throw some sata extension cord to verify if it is overheating or not.

Thanks
Gabriel Heifets

Fit-PC2/3/IntensePC support.

gritto
Posts: 3
Joined: Tue Dec 27, 2011 10:03 pm

Re: Ubuntu 11.04 server regular crashes

Post by gritto »

hello,
yes, we have verified by ourself the overheating problem.
We switch off a machine with tihis problem, put on a fridgidaire and wait 5 min. after it it was perfect.
We don't know how to find the external sata connector, do you have on your distributors? could you ship to Italy 60pcs of this cable?

we have 25 pcs on field, and 25 to be installed within the next month, you can understand, it is a very critical problem.
regards
Andrea

RKz
Posts: 81
Joined: Thu Apr 16, 2009 7:23 am

Re: Ubuntu 11.04 server regular crashes

Post by RKz »

Hi

Make sure you use 5400 rpm disks. Higher speed disks won't work reliably due to heat issues.
Richard Klingspetz
System Technology Sweden AB

gabrielh
Site Admin
Posts: 1260
Joined: Thu Jun 02, 2011 1:13 pm

Re: Ubuntu 11.04 server regular crashes

Post by gabrielh »

The problem can be caused if the fitpc placed in a small unventilated space, which prevents from the fitpc to cool down. If you are using a fast 7200 rpm disks, they produce more heat then 5400rpm one's. Please consider to switch to 5400 rpm disks if the fitpc forced to be placed in small, unventilated environment.
Gabriel Heifets

Fit-PC2/3/IntensePC support.

cbgoodbuddy
Posts: 21
Joined: Thu Mar 31, 2011 3:51 am

Re: Ubuntu 11.04 server regular crashes

Post by cbgoodbuddy »

Also consider getting the heat-sink accessory product. When I installed the heat-sink, the running temperature decreased by about 4-5C without active cooling.

Post Reply

Return to “Ubuntu 11.04”