Hello -
I am in the process of trying to determine the best parameters to use when creating and mounting an XFS file system on some new hardware we are going to be
using. My company has two separate applications that will be running on the same hardware (but on separate systems). One application is the PostgreSQL database and the other is a custom application that does writes to many (~1,000) relatively small (~672MB)
files concurrently.
I decided to use Bonnie++ to test a 17TB XFS file system, varying the mkfs.xfs and the mount parameters, in order to determine the optimal settings for our
new systems. I originally tried attaching my spreadsheet to this email, but I’m guessing your server doesn’t accept attachments. So,
I’ve
inserted my results as a table below; hopefully it will come through in a somewhat readable format.
If not, please send me your email and I’ll forward you the original spreadsheet.
For each set of parameters, I ran Bonnie++ 3 separate times and took the average of the results.
Also, a couple of things to note. First, we are already locked into this version of CentOS and the current kernel, so even though they are a bit dated, we
will not be able to upgrade to the latest kernel. Also, here are the release and version numbers of the XFS packages currently installed on these systems:
kmod-xfs-0.4-2
xfsprogs-devel-2.9.4-1.el5.centos
When I started specifying the sunit and swidth parameters, I expected to see the sequential block input and output to improve, but my final results didn’t really
show this (maybe my expectations were just wrong?). I was hoping that someone with
a lot more XFS expertise could take a look at my results and interpret them for me? Do my test results look accurate or is it possible that I setup my test incorrectly?
Based on the two applications that will be using an XFS file system, is there an optimal set of parameters that I can use for one or both applications? Any suggestions or recommendations would be greatly appreciated. Thanks in advance for your help in this
matter.
- Andrew
|
Data
|
Sequential Output
|
Sequential Input
|
Random Seeks
|
Sequential Create
|
Random Create
|
Number of Files
|
mkfs.xfs Options
|
Mount Options
|
|
Chunk Size
|
Per Char
|
Block
|
Rewrite
|
Per Char
|
Block
|
Create
|
Read
|
Delete
|
Create
|
Read
|
Delete
|
|
K/sec
|
%CP
|
K/sec
|
%CP
|
K/sec
|
%CP
|
K/sec
|
%CP
|
K/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
/sec
|
%CP
|
|
64352M
|
94013
|
99
|
472221
|
50
|
85811
|
11
|
80600
|
92
|
221248
|
14
|
377.6
|
0
|
2814
|
70
|
8036
|
92.7
|
12635
|
93.3
|
2924
|
71.7
|
7927
|
92.3
|
9819
|
80.7
|
10:688128:5120:1
|
-l size=16384b
|
rw
|
|
64352M
|
95284
|
99
|
450426
|
47.7
|
78142
|
10
|
74617
|
86.7
|
187983
|
12.3
|
341.7
|
0
|
2488
|
63
|
8672
|
99
|
11862
|
93
|
2669
|
67.3
|
7251
|
83
|
11165
|
87
|
10:688128:5120:1
|
-l size=16384b
|
rw,noatime,nodiratime,logbufs=8
|
|
64352M
|
90512
|
99
|
494516
|
51.3
|
84420
|
10.7
|
78464
|
89.7
|
207651
|
12
|
379.5
|
0.33
|
2694
|
64
|
9000
|
99
|
12772
|
97.3
|
3059
|
71
|
8480
|
93.3
|
10178
|
81.3
|
10:688128:5120:1
|
-l size=16384b
|
rw,noatime,nodiratime,logbufs=8,nobarrier
|
|
64352M
|
88295
|
99
|
501854
|
53
|
89128
|
11
|
80788
|
92.3
|
230101
|
16.7
|
368.4
|
0
|
3196
|
80.3
|
8552
|
98.3
|
12818
|
94.7
|
2693
|
64.3
|
7189
|
83.7
|
11396
|
90
|
10:688128:5120:1
|
-l size=128m
|
rw
|
|
64352M
|
91501
|
99
|
487799
|
51.3
|
91343
|
11.3
|
83336
|
95
|
236927
|
17
|
379.7
|
0
|
2852
|
72
|
8394
|
96.7
|
12879
|
95.7
|
2865
|
68
|
7957
|
91.3
|
11540
|
88
|
10:688128:5120:1
|
-l size=128m
|
rw,noatime,nodiratime,logbufs=8
|
|
64352M
|
96319
|
99
|
453202
|
47
|
82270
|
10.3
|
78626
|
89.3
|
204320
|
12.3
|
377.1
|
0.33
|
2948
|
70.7
|
8905
|
97.7
|
13072
|
97.7
|
3004
|
69.7
|
8324
|
92.3
|
12029
|
97.3
|
10:688128:5120:1
|
-l size=128m
|
rw,noatime,nodiratime,logbufs=8,nobarrier
|
|
64352M
|
92101
|
99
|
477882
|
50
|
90622
|
11
|
82742
|
94.7
|
239307
|
14.7
|
377.0
|
0
|
2800
|
69.3
|
8954
|
99
|
11597
|
85.3
|
2699
|
64.3
|
8836
|
100
|
10299
|
83.7
|
10:688128:5120:1
|
-l version=2,size=16384b
|
rw
|
|
64352M
|
98567
|
99
|
448664
|
46.7
|
81706
|
10
|
78214
|
90
|
217844
|
16
|
362.0
|
0
|
2529
|
64
|
8899
|
98
|
12754
|
95.7
|
2988
|
69
|
8419
|
93.3
|
9267
|
72.7
|
10:688128:5120:1
|
-l version=2,size=16384b
|
rw,noatime,nodiratime,logbufs=8
|
|
64352M
|
92854
|
99
|
482572
|
50.7
|
79940
|
9.67
|
76456
|
88.3
|
193196
|
14.3
|
344.5
|
0
|
2782
|
68
|
8925
|
99.3
|
12953
|
98.3
|
2904
|
67.7
|
8272
|
93.3
|
10709
|
86.7
|
10:688128:5120:1
|
-l version=2,size=16384b
|
rw,noatime,nodiratime,logbufs=8,nobarrier
|
|
64352M
|
96836
|
99
|
482944
|
51
|
92771
|
12
|
82157
|
94
|
240734
|
14
|
383.0
|
0.67
|
3176
|
79.3
|
8419
|
97
|
12932
|
96
|
2925
|
70.3
|
8572
|
99
|
11595
|
88.3
|
10:688128:5120:1
|
-l version=2,size=128m
|
rw
|
|
64352M
|
96595
|
99
|
493446
|
51.3
|
82685
|
10.3
|
78582
|
89.7
|
214803
|
12.7
|
366.2
|
0.33
|
3644
|
86.7
|
8805
|
99
|
13503
|
98.3
|
3316
|
77.3
|
7564
|
87
|
9005
|
69.3
|
10:688128:5120:1
|
-l version=2,size=128m
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
94856
|
99
|
516368
|
54
|
79747
|
10
|
75690
|
87.3
|
198909
|
13.7
|
352.6
|
0
|
3680
|
86.3
|
8800
|
99
|
13825
|
98.7
|
3652
|
85.3
|
7952
|
90.3
|
10891
|
85
|
10:688128:5120:1
|
-l version=2,size=128m
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
90933
|
99
|
463918
|
48.7
|
81266
|
10.3
|
76380
|
88.3
|
229959
|
14
|
372.3
|
0.33
|
2704
|
67.7
|
8670
|
99.3
|
12473
|
96.3
|
2878
|
70
|
8572
|
97.7
|
9065
|
70
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2
|
rw
|
|
64352M
|
89084
|
99
|
459402
|
49
|
80776
|
10.3
|
75495
|
86.7
|
230625
|
12.7
|
371.6
|
0.33
|
3173
|
74.7
|
7410
|
83.3
|
13247
|
96.7
|
4017
|
93.3
|
7778
|
87.3
|
10813
|
82.3
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
92994
|
99
|
486561
|
51.7
|
78986
|
9.67
|
76097
|
87.7
|
230976
|
12
|
379.1
|
0.33
|
3652
|
85.3
|
7255
|
81.7
|
11593
|
83.3
|
3941
|
90.7
|
8964
|
98.7
|
9354
|
71.7
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
89910
|
99
|
508021
|
53.3
|
81036
|
10
|
78428
|
90
|
214695
|
16.3
|
362.2
|
0
|
2891
|
72.3
|
8658
|
99.3
|
12888
|
95
|
2833
|
69
|
7999
|
93
|
11380
|
84.7
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2
|
rw
|
|
64352M
|
96944
|
99
|
487031
|
51
|
91412
|
11.3
|
82028
|
93.3
|
235545
|
14.3
|
374.9
|
0
|
3589
|
84.7
|
8579
|
99
|
13744
|
99
|
3622
|
84.3
|
7983
|
91
|
10703
|
82
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
92401
|
99
|
490172
|
51.7
|
78263
|
9.67
|
77018
|
88.7
|
235954
|
14.3
|
382.6
|
0.33
|
3612
|
84.3
|
8754
|
99
|
12172
|
89
|
3414
|
82.7
|
6170
|
71.3
|
9253
|
80.3
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
91268
|
99
|
464753
|
49.3
|
66828
|
8
|
67592
|
81
|
125015
|
10
|
362.6
|
0
|
2745
|
66.3
|
8755
|
99
|
9344
|
74.3
|
3198
|
73.7
|
7310
|
85
|
7252
|
60
|
10:688128:5120:1
|
-l version=2,size=16384b -d su=262144,sw=9
|
rw
|
|
64352M
|
95877
|
99
|
435503
|
46
|
60831
|
7
|
67252
|
80.3
|
125840
|
10
|
361.2
|
0
|
3044
|
75
|
8688
|
99.7
|
5479
|
53.3
|
3013
|
70.7
|
7009
|
81.3
|
11294
|
86.7
|
10:688128:5120:1
|
-l version=2,size=16384b -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
95787
|
99
|
481562
|
51
|
90843
|
11.3
|
82987
|
94.7
|
240849
|
13
|
378.4
|
0
|
3249
|
75.3
|
8124
|
89.7
|
9918
|
83.7
|
3582
|
85.3
|
7283
|
82.7
|
6402
|
50.3
|
10:688128:5120:1
|
-l version=2,size=16384b -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
88591
|
99
|
475663
|
50.7
|
83011
|
10.3
|
77000
|
89
|
220378
|
14
|
362.7
|
0
|
4197
|
97
|
8659
|
99.3
|
6089
|
47.3
|
3121
|
76
|
6257
|
71.3
|
10488
|
79.7
|
10:688128:5120:1
|
-l version=2,size=128m -d su=262144,sw=9
|
rw
|
|
64352M
|
91727
|
99
|
465144
|
49
|
80260
|
10
|
77585
|
89.7
|
233708
|
14.7
|
368.1
|
0
|
3739
|
87
|
8507
|
99.7
|
5478
|
42.7
|
3396
|
78.7
|
8602
|
94
|
8534
|
64.7
|
10:688128:5120:1
|
-l version=2,size=128m -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
95867
|
99
|
464367
|
49.3
|
80796
|
10
|
75411
|
87.7
|
218984
|
17
|
370.1
|
0
|
3671
|
87
|
8658
|
99
|
5500
|
50.7
|
3530
|
82.7
|
7536
|
85
|
8368
|
66.7
|
10:688128:5120:1
|
-l version=2,size=128m -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
98446
|
99
|
459868
|
48.3
|
78259
|
10
|
76096
|
88
|
226517
|
16
|
365.3
|
0
|
3281
|
76.7
|
6440
|
73
|
6267
|
47.3
|
2992
|
69.7
|
7664
|
89.3
|
7732
|
60.7
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
|
rw
|
|
64352M
|
91304
|
99
|
442525
|
47.3
|
82323
|
10.3
|
76163
|
88
|
229872
|
15.7
|
380.9
|
0
|
3015
|
71
|
7284
|
80.7
|
11545
|
89.3
|
3737
|
88
|
4466
|
50
|
9422
|
71
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
92167
|
99
|
396332
|
42
|
81941
|
10.3
|
76401
|
88
|
233618
|
15.7
|
382.5
|
0
|
3626
|
86.7
|
6662
|
75.3
|
11444
|
84
|
4003
|
91.7
|
8074
|
92.3
|
4023
|
31.7
|
10:688128:5120:1
|
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
64352M
|
92294
|
99
|
479220
|
50.3
|
81340
|
10.3
|
77236
|
89
|
235145
|
14.7
|
372.3
|
0.33
|
3854
|
89.7
|
8452
|
99
|
5212
|
43
|
3518
|
87
|
5118
|
60.7
|
12679
|
95
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
|
rw
|
|
64352M
|
95706
|
99
|
471999
|
49.7
|
82204
|
10.3
|
77009
|
88.7
|
225017
|
16.3
|
368.1
|
0
|
3740
|
88
|
8535
|
99.3
|
6571
|
49.7
|
3826
|
88.3
|
4362
|
49.7
|
11466
|
86.7
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k
|
|
64352M
|
96086
|
99
|
407269
|
43
|
82465
|
10
|
75962
|
88
|
236212
|
16.7
|
372.6
|
0.33
|
3793
|
88.3
|
8828
|
99.3
|
5416
|
45.3
|
4103
|
94.3
|
7496
|
85
|
8085
|
65
|
10:688128:5120:1
|
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
|
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MAX:
|
98567
|
99
|
516368
|
54
|
92771
|
12
|
83336
|
95
|
240849
|
17
|
383.0
|
0.67
|
4197
|
97
|
9000
|
99.7
|
13825
|
99
|
4103
|
94.3
|
8964
|
100
|
12679
|
97.3
|
|
|
|
|
MIN:
|
88295
|
99
|
396332
|
42
|
60831
|
7
|
67252
|
80.3
|
125015
|
10
|
341.7
|
0
|
2488
|
63
|
6440
|
73
|
5212
|
42.7
|
2669
|
64.3
|
4362
|
49.7
|
4023
|
31.7
|
|
|
|
|
AVG:
|
93504
|
99
|
470040
|
49.5
|
82048
|
10.2
|
77414
|
89.2
|
217398
|
14.2
|
369.9
|
0.12
|
3239
|
77.7
|
8376
|
95
|
10599
|
80.5
|
3281
|
77.4
|
7513
|
85.5
|
9807
|
76.6
|
|
|
|
|
MED:
|
92924
|
99
|
473942
|
50
|
81824
|
10.3
|
77014
|
88.8
|
228195
|
14.3
|
372.0
|
0
|
3186
|
76
|
8658
|
99
|
12017
|
91.2
|
3160
|
74.8
|
7853
|
89.8
|
10239
|
81
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Controller
|
RAID
|
# Disks
|
Stripe Size
|
Type
|
Disk Size
|
FS Size
|
OS
|
Kernel
|
Motherboard
|
Chipset
|
RAM
|
Processors
|
|
|
|
3ware 9650SE-12M
|
6
|
11
|
256 KB
|
SATA
|
1.82 TB
|
17 TB
|
CentOS 5.3
|
2.6.18-128.el5
|
Supermicro X8DTU
|
Intel 5520 (Tylersburg)
|
32GB PC3-8500 1066MHz
|
Dual Intel Xeon E5540 @2.53GHz
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This email and any attachments may be confidential and are intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient of this email, the following stipulations govern the use of this information: You may not take any action based upon its contents. You may not copy or show this message or attachments to anyone. You should contact the sender
and subsequently delete this message and all attachments.
Any views or opinions expressed are solely those of the author and do not necessarily represent those of Special Operations Technology, Incorporated.
While antivirus software has been applied, you should perform due diligence to check this email and attachments for the presence of viruses. No warranties or assurances are made in relation to the safety and content of this email and attachments. Special Operations
Technology, Incorporated accepts no liability for any damages caused by any virus transmitted by or contained in this email and attachments.
No liability is accepted for any consequences arising from this email transmission whatsoever.
Special Operations Technology, Incorporated is a premier IT professional services firm focused in the government and law enforcement space.
|
