I am writing a console application to manage anyting up to 100k processes (or as many as there is room for in memory) and I’m using QProcess to set up two-way communication with each process.
When I use any of the “waitFor..” methods in QProcess, such as “waitForStarted()” or “waitForReadyRead()”, I get a buffer overflow error from Qtcore after starting around 200 processes.
When I don’t use any of these methods, I am able to start 10k processes easily. I am also able to connect to the signals of these processes, so instead of explicitly calling “waitForReadyRead()” I can wait for the process to fire the signal “readyRead()”, and then read the process output without problems.
In other words, I’ve found a workaround (or maybe the proper solution), but it worries me that I have to be picky about which methods to use to avoid ugly crashes.
Any insight would be appreciated.
PS:
On Linux and Mac the maximum open files pr. user has to be changed first, otherwise that will cause another (understandable) crash. In my case (on Ubuntu 12.4) I do this with `ulimit -n 102400`
Example code:
#include <iostream>
#include <QProcess>
#include <vector>
using namespace std;
vector<QProcess*> procs;
QString command="./myProcess";
QStringList args;
int process_count=200;
int main(int argc, char** argv){
if(argc>1)
process_count=atoi(argv[1]);
cout << "Setting up " << process_count << " processes" << endl;
/*
Create
*/
for(int i=0;i<process_count; i++)
procs.push_back(new QProcess);
cout << "Created processes" << endl;
/*
Start
*/
vector<QProcess*>::iterator it;
for(it=procs.begin(); it!=procs.end(); it++)
(*it)->start(command,args);
cout << "All started. Checking their output..." << endl ;
/*
Read
*/
for(it=procs.begin(); it!=procs.end(); it++){
(*it)->waitForStarted(); //~200 => CRASH
(*it)->waitForReadyRead(); //~200 => CRASH
}
cout << "All processes responded. " << endl;
/*
Kill
*/
for(it=procs.begin(); it!=procs.end(); it++)
(*it)->kill();
cout << "All killed. Done. "<< endl;
}
Backtrace of core dump:
*** buffer overflow detected ***: ./qprocess_testing terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f02694ea807]
/lib/x86_64-linux-gnu/libc.so.6(+0x109700)[0x7f02694e9700]
/lib/x86_64-linux-gnu/libc.so.6(+0x10a7be)[0x7f02694ea7be]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(+0x1554d4)[0x7f0269e0d4d4]
./qprocess_testing[0x400e35]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f026940176d]
./qprocess_testing[0x400fa9]
And the code of the dummy process “myProcess”:
#include <iostream>
using namespace std;
void work(){
for(int i=0;i<200000000;i++){
if(false){ //Dirty hack to avoid compiler optimization
cout << "No way: " << i << endl;
}
}
return;
}
int main(){
sleep(1);
cout << "!";
char c;
while(true){
cin >> c;
work();
cout <<++c;
}
}
↧