I struggled with this as well until I found out the limits.conf entries don't apply to systemd managed services. If you create a /etc/systemd/system/rabbitmq-server.service.d directory and place a limits.conf file in there with contents similar to [Service] LimitNOFILE=4096 Then reload + restart systemctl daemon-reload systemctl restart rabbitmq-server https://fedoraproject.org/wiki/Systemd#How_do_I_customize_a_unit_file.2F_... On Thu, Nov 19, 2015 at 10:43 PM, Erich Weiler <weiler(a)soe.ucsc.edu <mailto:weiler@soe.ucsc.edu>> wrote: Hi Y'all, I'm sure someone has encountered this issue... basically my rabbitmq instance on my controller node is running out of file descriptors, this is on RHEL 7. I've upped the max file descriptors (nofile) to 1000000 in /etc/security/limits.conf, and my sysctl limit for file descriptors is equally huge. Yet, I can't get my rabbitmq process to get it's limit's past 1000 or so: [root@os-con-01 ~]# ps -afe | grep rabbit rabbitmq 4989 1 4 16:42 ? 00:07:10 /usr/lib64/erlang/erts-5.10.4/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@os-con-01 -boot start_sasl -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,&quot;/var/log/rabbitmq/rabbit(a)os-con-01.log&quot;} -rabbit sasl_error_logger {file,&quot;/var/log/rabbitmq/rabbit(a)os-con-01-sasl.log&quot;} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@os-con-01-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@os-con-01" -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672 rabbitmq 5004 1 0 16:42 ? 00:00:00 /usr/lib64/erlang/erts-5.10.4/bin/epmd -daemon rabbitmq 5129 4989 0 16:42 ? 00:00:00 inet_gethost 4 rabbitmq 5130 5129 0 16:42 ? 00:00:00 inet_gethost 4 root 17470 17403 0 19:34 pts/0 00:00:00 grep --color=auto rabbit [root@os-con-01 ~]# cat /proc/4989/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 127788 127788 processes Max open files 1024 4096 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127788 127788 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us [root@os-con-01 ~]# This is causing huge problems in my OpenStack cluster (Kilo Release). I've read that you can set this limit in /etc/rabbitmq/rabbitmq-env.conf or /etc/rabbitmq/rabbitmq.config but no matter what I do there I get nothing, after restarting rabbitmq many times. Does this have something to do with systemd? [root@os-con-01 ~]# rabbitmqctl status Status of node 'rabbit@os-con-01' ... [{pid,4989}, {running_applications,[{rabbit,"RabbitMQ","3.3.5"}, {os_mon,"CPO CXC 138 46","2.2.14"}, {mnesia,"MNESIA CXC 138 12","4.11"}, {xmerl,"XML parser","1.3.6"}, {sasl,"SASL CXC 138 11","2.3.4"}, {stdlib,"ERTS CXC 138 10","1.19.4"}, {kernel,"ERTS CXC 138 10","2.16.4"}]}, {os,{unix,linux}}, {erlang_version,"Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:32:32] [async-threads:30] [hipe] [kernel-poll:true]\n"}, {memory,[{total,645523200}, {connection_procs,32257624}, {queue_procs,48513416}, {plugins,0}, {other_proc,15448376}, {mnesia,1209984}, {mgmt_db,0}, {msg_index,292800}, {other_ets,1991744}, {binary,517865992}, {code,16698259}, {atom,602729}, {other_system,10642276}]}, {alarms,[]}, {listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,13406973132}, {disk_free_limit,50000000}, {disk_free,37354610688}, {file_descriptors,[{total_limit,924}, <----- ?????? {total_used,831}, {sockets_limit,829}, {sockets_used,829}]}, {processes,[{limit,1048576},{used,8121}]}, {run_queue,0}, {uptime,10537}] ...done. Anyone know how to get the file descriptor limits up for rabbitmq? I've only got like 40 nodes in my OpenStack cluster, and it's choking, and I need to add several hundred more nodes... Any help much appreciated!!! I looked around the list and couldn't find anything on this, and I've RTFM'd as much as I could... cheers, erich _______________________________________________ Rdo-list mailing list Rdo-list(a)redhat.com <mailto:Rdo-list@redhat.com> https://www.redhat.com/mailman/listinfo/rdo-list To unsubscribe: rdo-list-unsubscribe(a)redhat.com <mailto:rdo-list-unsubscribe@redhat.com>