All Questions
Tagged with performance io
99
questions
1
vote
1
answer
55
views
Disk saturated by mariadb on CentOS 7, blocks everything
For the past few days I have been having a problem with a webserver running CentOS 7. Every 13-14 minutes some operation (which I identified with mariadb using iostat) seems to completely saturate the ...
2
votes
1
answer
1k
views
contradictory results dd and iostat, ext4 and xfs, on EBS volumes
I'm seeing what seems to me to be contradictory results when looking at disk performance with dd versus iostat on two hosts (EC2 instances with an EBS drive). The hosts are identical except that one ...
- 1,213
9
votes
2
answers
8k
views
Why do sequential writes have better performance than random writes on SSDs?
An LBA (logical block addresses) is a mapping table implemented in the FTL to match between logical and physical pages/blocks in SSDs, my guess is that most SSDs (at least when they are empty) keeps ...
- 78.1k
1
vote
0
answers
66
views
Couchdb 3.1.1 increases disk latency gradually over weeks
I've been dealing with this issue for weeks now.
I have the followin scenario:
couchdb2.3.1-A <===> couchdb2.3.1-B <===> couchdb3.1.1-A <===> couchdb3.1.1-B
where <===> ...
-1
votes
1
answer
427
views
Simulate some process IO with fio or other tool
Is there any way to track a specific process and get stats for disk IO such as queue depth, total reading/writing threads, percent of reads/writes and so on?
The main goal is to use all that ...
- 525
154
votes
13
answers
75k
views
Are networks now faster than disks?
This is a software design question
I used to work on the following rule for speed
cache memory > memory > disk > network
With each step being 5-10 times the previous step (e.g. cache ...
5
votes
1
answer
3k
views
Linux Read-Ahead Downsides
Are there any downsides to raising the size of the read-ahead cache?
On our farm, we're currently running at 256, and upon raising that higher, we are seeing significant throughput gains.
[root@...
- 191
2
votes
1
answer
404
views
About IOPS of a RAID5 disk array without cache
we are using a 530-8i RAID adapter,which is none-cache supported, on a Lenovo ThinkSystem SR590 server.And we have 4 HDD(600GB/10K RPM/SAS) in the array.
I read an article about calculating the disk/...
- 48.5k
2
votes
0
answers
612
views
Access disk io data in linux without tools like iotop/iostat to find processes or files with IO
We have regularly high load average. With the help of a zabbix agent, I could narrow it down to be mainly because of cpu iowait (and lastly disk io). I am not allowed to install any additional ...
- 121
0
votes
1
answer
277
views
Is RAID 6 a good choice regarding performance on server used for computation?
We are operating a server for doing computations with large data sets (100GB - 5TB) for a small group of researchers (10-20 people). At the moment we are running 4x8TB spinning drives in a RAID 6 ...
- 101k
11
votes
4
answers
12k
views
linux: how to simulate hard disk latency? I want to increase iowait value without using a cpu power
Is it possible to set an arbitrary delay for I/O operations?
I would like to learn how do iowait and disk %util impact the performance. There are a lot of tools to stress the disk (bonnie++ etc.) but ...
- 372
2
votes
1
answer
104
views
MYSQL vs. InnoDB [closed]
mysqltuner output.
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +...
- 2,483
1
vote
0
answers
140
views
disk I/O latency spikes (contains log files from amplify)
Ubuntu 18.04 LTS server, disk is ssd. Application stack is: mongodb, node, nignx and some python.
Server is up 1 month and already had 3 OS disk fails where Ubuntu would go in read only mode. After ...
1
vote
0
answers
674
views
How does avgqu-sz size in iostat output relate to r_await and w_await?
Can someone explain what does avgqu-sz mean in the output of iostat? The manual page states like this.
avgqu-sz:
The average queue length of the requests that were issued to the device.
Can ...
- 333
1
vote
2
answers
6k
views
RAID 50 vs RAID 10 for performance?
I have a server with 6 SSD's and a raid controller card that supports both RAID 10 and RAID 50 the plan is to use this as our Build server. It will be pulling in NPM Packages and many small code files,...
- 32.4k
0
votes
1
answer
155
views
jdb2 spike every hour for few seconds
I get IO hang up every hour on the exact same time on my Ubuntu 18.04 server.
iotop indicate that jbd2 spike to araong 30% around this time.
The hangup last only few seconds.
I started noticing this ...
2
votes
1
answer
276
views
Improving I/O rates for EBS snapshot backed volumes
I'm working with a system that takes a set of 42 rotating daily EBS snapshots of each of its numerous (40) volumes for disaster recovery purposes. EBS volumes are aggregated into a RAID volume. A set ...
- 22.7k
5
votes
1
answer
790
views
Misunderstanding about Page Cache and dirty_background_bytes
I've been looking at this for a while now and things aren't lining up with my expectations, but I don't know if it's because something is off, or if my expectations are wrong.
So, I've got a system ...
- 48.5k
4
votes
1
answer
2k
views
KVM bad IO sync performance in Linux guest
I'm setting up virtualized Ubuntu 15.10 guest inside Ubuntu 15.04 server using libvirt and KVM.
Storage in host system:
# mount | grep ext4
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,...
CommunityBot
- 1
2
votes
4
answers
8k
views
How to reduce mysql disk io?
mysql> explain SELECT *
-> FROM (`items`)
-> WHERE `states` = 'A'
-> ORDER BY `updated_date` desc
-> LIMIT 20520, 40;
+----+-------------+--------+--------+--------------...
- 143
1
vote
1
answer
215
views
Bandwidth and IO issues when running hardware raid and zfs
I've inherited a setup using Linux Ubuntu 14, a megaraid_sas, and ZFS.
We're getting some performance problems (never being able to fully utilize the 6GB bandwidth from the raid) and I'm curious if ...
- 717
39
votes
5
answers
81k
views
How to identify I/O bottlenecks on a Linux server?
How do you determine that the performance of your Linux server is I/O bound and, perhaps more importantly, what process or processes are casuing the problem?
3
votes
2
answers
881
views
How does I/O performance on Intel E5 procs change with and without QPI
I am in the process of specifying a low-mid range server (which we will purchase multiples of). The application is essentially a software router, so it is very NIC-based I/O intensive. Processor ...
- 245k
8
votes
2
answers
19k
views
what is the relation between block size and IO?
I have been reading about disk recently which led me to 3 different doubts. And I am not able to link them together. Three different terms I am confused with are block size, IO and Performance.
I ...
- 101
1
vote
0
answers
451
views
Give read operations priority over write
I'm working on a system where I have a program that outputs high quality videos for a tv connected to an HDMI (raw 1080p data for instance) and I'm having an issue that when I copy some files to my ...
- 41
3
votes
1
answer
988
views
Linux: how can I determine if an application is doing sequential or random disk I/O?
Everyone knows that databases tend to do lots of small random I/O while big data things like Kafka tend to do large sequential I/O, but if I'm approaching this as a sysadmin without making assumptions,...
- 2,655
2
votes
0
answers
1k
views
Reason for high CPU time when executing fio
After weeks of troubleshooting some performance issues I have on my VPS running Ubuntu 12.04 LTS, I am getting closer to the conclusion that the problem is related to IO.
When I have a considerable ...
- 1
6
votes
1
answer
917
views
RAID6 resync with fast writes but slow reads
I'm using Debian Jessie.
# uname -a
Linux host 4.9.0-0.bpo.3-amd64 #1 SMP Debian 4.9.30-2+deb9u5~bpo8+1 (2017-09-28) x86_64 GNU/Linux
And have setup a RAID6.
# cat /proc/mdstat
Personalities : [...
- 10.4k
1
vote
1
answer
3k
views
Google compute: Why persistent ssd disks are slower than persistent standard disks?
I ran several IO benchmarks and I cannot understand how pd-ssd offers a worse IO throughtput than pd-standard.
In my tests I boot a VM with a bootable disk pd-ssd of 30 or 250GB, and I still get ...
google-cloud-platform
3
votes
1
answer
4k
views
What does the nomerge mean in Linux system?
We can see some parameters by issuing "iostat -x -d", the rrqm/s and wrqm/s stand for the merge for I/O request per second. And there is a kernel parameter controlling the I/O merge:
/sys/class/block/...
- 32.4k
3
votes
2
answers
2k
views
Disk IO Rate: What causes it and what does it mean to me?
I just got a mail from Linode saying that my account "has exceeded the notification threshold (1000) for disk io rate by averaging 1017.36 for the last 2 hours. "
My question is just out of curious:
...
- 1
3
votes
0
answers
54
views
Do I need to create multiple filesystems on a virtual server to increase I/O performance?
I want to install Oracle on a virtual server. I don't really know the technology behind, my company just gave me a virtual server with multiple (virtual) disks.
I have the possibility to divide those ...
- 83
6
votes
3
answers
5k
views
Massive performance degradation on sustained sequential write
I'm in the process of migrating data into LUKS partitions. Now that the operating system drive is running off LUKS, I tried to start migrating the data drives. Then the server stopped responding.
...
- 1,070
10
votes
2
answers
10k
views
MySQL extremely slow on very simple SELECT queries
We have a simple web application running on a virtual machine that saves its data in a MySQL 5.5 database with the InnoDB engine. Everything worked fine for around three years, but suddenly it became ...
- 141
5
votes
2
answers
2k
views
Tuning sequential disk reads for performance
I am reading a large file sequentially from the disk and trying to understand the iostat output while the reading is taking place.
Size of the file : 10 GB
Read Buffer : 4 KB
Read ahead (/sys/block/...
CommunityBot
- 1
7
votes
1
answer
2k
views
How efficient is the tac command on large files
The taccommand (catreversed) can be used to read a file backwards, just like cat reads it rom the beginning. I wonder, how efficient this is. Does it have to read the whole file from the beginning and ...
- 946
0
votes
2
answers
2k
views
Poor Write I/O Performance on HP ML110 G9 Server With ESXi 6.0U2
hi everyone i need a little bit of help since i cant find a solution,
i have a HP ML110 G9 Server with:
8gb Ram
2 HDD of 2tb Each one and one SSD of 250 GB
i have installed ESXi 6.0u2 HP Customized....
3
votes
6
answers
20k
views
Show IO on Netapp
I think I might be hitting the IO limits of what my Netapp can deliver, as I have been adding more servers to my cluster and iowait has gone up on each server.
However, how do I quantify this? How ...
CommunityBot
- 1
1
vote
1
answer
1k
views
I/O scheduling on LVM on dm-crypt
I have the following setup:
Stock Debian stable (Linux kernel 3.16.7) running on Dell PowerEdge R320
Two SATA hard drives in RAID1 on a PERC H310 Mini controller, visible as /dev/sda
dm-crypt target ...
- 48.5k
10
votes
6
answers
14k
views
How to identify heavy write to disk?
I have this problem with server running CakePHP application. The server is insanely slow, I first thought that it's application problem, but then I found constant 5-6MB/s write to disk.
What is the ...
- 2,050
2
votes
1
answer
420
views
How can I determine which files have high I/O in Windows?
In trying to reduce the storage bottleneck on a Windows server (2008 R2) I need to find the files that have the highest I/O over a long period of time (aka 'hot' files) so I can move those to SSD ...
- 398
4
votes
1
answer
6k
views
Filesystem Performance for LUKS Encrypted Volumes?
As you can format a LUKS volume with any filesystem you want, are certain filesystems better-suited to IO performance against an encrypted filesystem? Specifically, I'm wondering about a loop device ...
- 245k
0
votes
0
answers
620
views
How to find the bottleneck for slow rsync
I want to find the bottleneck for a slow rsync.
I see these parts (on byte flow order):
IO on source
CPU on source
Network
CPU on destination
IO on destination
How can I debug this?
- 3,603
1
vote
0
answers
39
views
Long Response Times for Reads via FCoE
We're getting some intriguing performance with our new FCoE environment, and I was hoping that people could let me know if what we're seeing is expected behaviour.
Our switching environment consists ...
- 842
1
vote
1
answer
114
views
How to check what is causing performance issue
[Edit] This question is different than what is suggested by the moderator, I am trying to find out what could have caused the API to become slow because of the few lines of codes. I am not trying to ...
- 945
0
votes
3
answers
2k
views
I/O and RAM limitations are important for Hadoop performance. But is disk speed related to I/O?
Hortonworks says this: "Most often performance of a Hadoop cluster will not be constrained by disk speed – I/O and RAM limitations will be more important." *
How is disk speed not related to I/O ...
- 1,466
1
vote
0
answers
269
views
Redis parent process smashing disk despite appendfsync = no
We are seeing loads of IO from redis-server despite our save operations being set to occur once every 15 minutes only.
The workload is a "full page cache"; we store blobs of compressed HTML. The HTML ...
- 457
3
votes
4
answers
3k
views
Azure IO performance
We have a number of Windows Server virtual machines that we use for .NET and Java development and QA. I find Azure attractive for the obvious reasons but mainly for the opportunity to reduce non-...
- 111
2
votes
3
answers
5k
views
High disk I/O activity in CentOS server
I have about 16 websites in a CentOS dedicated, and I am having some problems on high traffic hours, it seems to be a high disk I/O activity causing a general slowdown.
I've installed atop and this ...
CommunityBot
- 1
0
votes
1
answer
44
views
Is "'Light-Speed' Disk Access" really a thing? [closed]
I just saw an advert for Hyve hosting on You Tube. One statement made me curious, they said they offer "'Light-Speed' Disk Access":
Is this just a fancy way of saying they have optical technology, ...
CommunityBot
- 1