Hello, I am trying to install OpenStack PackStack in DELL360 server but getting issues. Details as follows: Hardware: DELL360 PowerEdge Processor: Intel XEON E5-2699 v4 @ 2.20 Ghz [2 physical CPU's] Memory: 512 GB Disk type: Intel SSD's OS tried: Cent OS 7 (2003,2009) , Cent OS 8.3 RDO repo's tried: Victoria,Ussuri,Train

Errors statements: - Command Exceeded timeout at the end of pp execution - Systemd start for openstack-nova-scheduler failed

From there it could have some hint.

Hello Yatin, On Wed, 10 Feb 2021 at 10:51, YATIN KAREL <yatinkarel(a)gmail.com&gt; wrote: > > Hi Amey, > > On Wed, Feb 10, 2021 at 9:37 AM Amey Abhyankar <sco1984(a)gmail.com&gt; wrote: >> >> Hello, >> >> I am trying to install OpenStack PackStack in DELL360 server but getting issues. >> >> Details as follows: >> >> Hardware: DELL360 PowerEdge >> Processor: Intel XEON E5-2699 v4 @ 2.20 Ghz [2 physical CPU's] >> Memory: 512 GB >> Disk type: Intel SSD's >> OS tried: Cent OS 7 (2003,2009) , Cent OS 8.3 >> RDO repo's tried: Victoria,Ussuri,Train > > Victoria, Ussuri should install fine over CentOS8, Train should install fine on both CentOS7 and CentOS8. >> >> Errors statements: >> - Command Exceeded timeout at the end of pp execution >> - Systemd start for openstack-nova-scheduler failed > > Have you checked the nova-scheduler logs(/var/log/nova/nova-scheduler.log) to see why it failed and also the pp execution log to see where it's stuck? From there it could have some hint. > Need to see more logs/info to understand the issue, starting with:- > - nova-scheduler log > - packstack install logs > - output of "dnf repolist" > - answer file and installation commands used > - output of "ip a" > - output of "getenforce" I tried with Rocky release yesterday. I saw file descriptor errors in MySQL DB log. I am using CentOS 7 (2003 version) MySQL DB kept eating 99% CPU for an hour+ eventually packstack installation failed Errors from the log file = -------------------------------------------------------------------------------- Warning: Unknown variable: '::nova::vncproxy::host'. at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/vncproxy/common.pp:31:5 Warning: Unknown variable: '::nova::vncproxy::vncproxy_protocol'. at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/vncproxy/common.pp:36:5 Warning: Unknown variable: '::nova::vncproxy::port'. at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/vncproxy/common.pp:41:5 Warning: Unknown variable: '::nova::vncproxy::vncproxy_path'. at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/vncproxy/common.pp:46:5 Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at ["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/placement.pp", 101]:["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/manifests/10.200.104.32_compute.pp", 26] (at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation') Warning: Scope(Class[Nova::Placement]): The os_region_name parameter is deprecated and will be removed \ in a future release. Please use region_name instead. Warning: Unknown variable: '::nova::compute::pci_passthrough'. at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/compute/pci.pp:19:38 Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/compute/libvirt.pp", 278]:["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/manifests/10.200.104.32_compute.pp", 32] (at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation') Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ip_address instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match... . at ["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/nova/manifests/compute/neutron.pp", 31]:["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/manifests/10.200.104.32_compute.pp", 47] (at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation') Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Array instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match... . at ["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/neutron/manifests/init.pp", 492]:["/var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/manifests/10.200.104.32_compute.pp", 48] (at /var/tmp/packstack/2f41026548a941369ed4234e12e1a3a3/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation') Warning: ModuleLoader: module 'vswitch' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules (file & line not available) Warning: ModuleLoader: module 'mysql' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules (file & line not available) Notice: /Stage[main]/Ceilometer::Deps/Anchor[ceilometer::service::end]: Triggered 'refresh' from 1 events Notice: /Stage[main]/Nova::Deps/Anchor[nova::config::end]: Triggered 'refresh' from 21 events Notice: /Stage[main]/Nova::Deps/Anchor[nova::service::begin]: Triggered 'refresh' from 1 events Error: Systemd start for openstack-nova-compute failed! journalctl log for openstack-nova-compute: -- Logs begin at Wed 2021-02-10 20:08:31 +0530, end at Wed 2021-02-10 23:34:41 +0530. -- Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: openstack-nova-compute.service: main process exited, code=exited, status=1/FAILURE Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: Failed to start OpenStack Nova Compute Server. Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: Unit openstack-nova-compute.service entered failed state. Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: openstack-nova-compute.service failed. Error: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Service[nova-compute]/ensure: change from stopped to running failed: Systemd start for openstack-nova-compute failed! journalctl log for openstack-nova-compute: -- Logs begin at Wed 2021-02-10 20:08:31 +0530, end at Wed 2021-02-10 23:34:41 +0530. -- Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: openstack-nova-compute.service: main process exited, code=exited, status=1/FAILURE Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: Failed to start OpenStack Nova Compute Server. Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: Unit openstack-nova-compute.service entered failed state. Feb 10 23:34:41 prod-openstack-svr1 systemd[1]: openstack-nova-compute.service failed. Notice: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Service[nova-compute]: Triggered 'refresh' from 1 events Notice: /Stage[main]/Nova::Deps/Anchor[nova::service::end]: Dependency Service[nova-compute] has failures: true Warning: /Stage[main]/Nova::Deps/Anchor[nova::service::end]: Skipping because of failed dependencies Notice: /Stage[main]/Nova::Logging/File[/var/log/nova/nova-manage.log]: Dependency Service[nova-compute] has failures: true Warning: /Stage[main]/Nova::Logging/File[/var/log/nova/nova-manage.log]: Skipping because of failed dependencies Notice: /Stage[main]/Nova/Exec[networking-refresh]: Dependency Service[nova-compute] has failures: true Warning: /Stage[main]/Nova/Exec[networking-refresh]: Skipping because of failed dependencies Notice: /Stage[main]/Nova::Cell_v2::Discover_hosts/Exec[nova-cell_v2-discover_hosts]: Dependency Service[nova-compute] has failures: true Warning: /Stage[main]/Nova::Cell_v2::Discover_hosts/Exec[nova-cell_v2-discover_hosts]: Skipping because of failed dependencies Notice: /Stage[main]/Neutron::Agents::Ml2::Ovs/Neutron::Plugins::Ovs::Bridge[extnet:br-ex]/Vs_bridge[br-ex]/external_ids: external_ids changed '' to 'bridge-id=br-ex' -------------------------------------------------------------------------------- >

> Installation type's tried: answer files, advanced networking > > Have you also tried out basic packstack installation(sudo packstack --allinone) to ensure it's not related to custom changes in the answer file. >> >> Yum update: tried with yum update & without yum update. > > Yes it's better to do a yum update before installation to keep the system up to date with latest CentOS packages. >> >> >> I did the same installation on Supermicro server with Intel XEON >> Silver 4110 @ 2.10 Ghz using the same set of software's & installation >> methods. No issues found. >> > Considering you have not faced issues in other hardware and different releases failed for you on DELL360 PowerEdge seems more like hardware related. > >> Anybody encountered similar issues? Thanks. >> >> Regards, >> Amey. >> _______________________________________________ >> users mailing list >> users(a)lists.rdoproject.org >> http://lists.rdoproject.org/mailman/listinfo/users >> >> To unsubscribe: users-unsubscribe(a)lists.rdoproject.org > > > > Thanks and Regards > Yatin Karel

Hello Yatin, On Thu, 11 Feb 2021 at 14:17, YATIN KAREL <yatinkarel(a)gmail.com&gt; wrote: > > Hi Amey, >> >> Hello, >> >> >> >> I am trying to install OpenStack PackStack in DELL360 server but getting issues. >> >> >> >> Details as follows: >> >> >> >> Hardware: DELL360 PowerEdge >> >> Processor: Intel XEON E5-2699 v4 @ 2.20 Ghz [2 physical CPU's] >> >> Memory: 512 GB >> >> Disk type: Intel SSD's >> >> OS tried: Cent OS 7 (2003,2009) , Cent OS 8.3 >> >> RDO repo's tried: Victoria,Ussuri,Train >snip< > Instead of trying all releases, let's stick to one and collect logs to understand the issue, I would suggest you to try packsack default installation with openstack-victoria release, and if it fails >collect logs that i asked in previous mail. When there are no proper errors, it is difficult to troubleshoot. Finally the issue is resolved. It was due to the warning of SMT [1] while installing CentOS. I disabled the logical CPU's in BIOS > Installed CentOS 7 > Installed packstack Train > All good. But now I have half of the CPU threads available for use. Very strange issue BTW of SMT + CentOS kernel + PackStack [1] access.redhat.com/solutions/rhel-smt >

> Also you can connect on Freenode IRC channel #rdo where you may get more early responses. > >> >> Installation type's tried: answer files, advanced networking >> > >> > Have you also tried out basic packstack installation(sudo packstack --allinone) to ensure it's not related to custom changes in the answer file. >> >> >> >> Yum update: tried with yum update & without yum update. >> > >> > Yes it's better to do a yum update before installation to keep the system up to date with latest CentOS packages. >> >> >> >> >> >> I did the same installation on Supermicro server with Intel XEON >> >> Silver 4110 @ 2.10 Ghz using the same set of software's & installation >> >> methods. No issues found. >> >> >> > Considering you have not faced issues in other hardware and different releases failed for you on DELL360 PowerEdge seems more like hardware related. >> > >> >> Anybody encountered similar issues? Thanks. >> >> >> >> Regards, >> >> Amey. >> >> _______________________________________________ >> >> users mailing list >> >> users(a)lists.rdoproject.org >> >> http://lists.rdoproject.org/mailman/listinfo/users >> >> >> >> To unsubscribe: users-unsubscribe(a)lists.rdoproject.org >> > >> > >> > >> > Thanks and Regards >> > Yatin Karel > > > > Thanks and Regards > Yatin Karel

Hello, Well it doesn't stop here unfortunately. Installation is successful but If I do any actions I get errors. When I am creating a instance using cirros-0.5.1-x86_64-disk.img from Horizon dashboard, I am getting following error = Block Device Mapping is Invalid: failed to get image 11568f06-7513-4750-8be6-e531c9829a01. (HTTP 400) (Request-ID: req-c208f78c-d649-482e-8a0c-fc950b127ebe) Also, I am getting error while creating key pair from horizon dashboard. Any hints how to fix this? [root@prod-openstack-svr ~(keystone_admin)]# openstack image list +--------------------------------------+--------+--------+ | ID | Name | Status | +--------------------------------------+--------+--------+ | 11568f06-7513-4750-8be6-e531c9829a01 | cirros | active | +--------------------------------------+--------+--------+ [root@prod-openstack-svr ~(keystone_admin)]# nova service-list +--------------------------------------+----------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | Forced down | +--------------------------------------+----------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+ | 460f8560-476c-4b13-bab9-48db345c51bf | nova-conductor | prod-openstack-svr.datametica.com | internal | enabled | up | 2021-02-16T03:07:21.000000 | - | False | | c1a3c31e-f67b-49f0-96dc-b9870e81e1c4 | nova-scheduler | prod-openstack-svr.datametica.com | internal | enabled | up | 2021-02-16T03:07:27.000000 | - | False | | c1332b4c-6b7f-4927-bb76-83cae19fa51b | nova-compute | prod-openstack-svr.datametica.com | nova | enabled | up | 2021-02-16T03:07:29.000000 | - | False | +--------------------------------------+----------------+-----------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+ [root@prod-openstack-svr ~(keystone_admin)]# systemctl status openstack-cinder-api ● openstack-cinder-api.service - OpenStack Cinder API Server Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-api.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2021-02-12 19:13:10 IST; 3 days ago Main PID: 2254 (cinder-api) Tasks: 45 CGroup: /system.slice/openstack-cinder-api.service ├─ 2254 /usr/bin/python2 /usr/bin/cinder-api --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/api.log ├─12355 /usr/bin/python2 /usr/bin/cinder-api --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/api.log ├─12356 /usr/bin/python2 /usr/bin/cinder-api --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/api.log [root@prod-openstack-svr ~(keystone_admin)]# cinder service-list +------------------+------------------------+------+---------+-------+----------------------------+---------+-----------------+---------------+ | Binary | Host | Zone | Status | State | Updated_at | Cluster | Disabled Reason | Backend State | +------------------+------------------------+------+---------+-------+----------------------------+---------+-----------------+---------------+ | cinder-backup | prod-openstack-svr | nova | enabled | up | 2021-02-16T03:08:25.000000 | - | - | | | cinder-scheduler | prod-openstack-svr | nova | enabled | up | 2021-02-16T03:08:25.000000 | - | - | | | cinder-volume | prod-openstack-svr@lvm | nova | enabled | up | 2021-02-16T03:08:24.000000 | - | - | up | +------------------+------------------------+------+---------+-------+----------------------------+---------+-----------------+---------------+ [root@prod-openstack-svr ~(keystone_admin)]# service neutron-server status Redirecting to /bin/systemctl status neutron-server.service ● neutron-server.service - OpenStack Neutron Server Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2021-02-12 19:13:40 IST; 3 days ago Main PID: 2291 (/usr/bin/python) Tasks: 92 CGroup: /system.slice/neutron-server.service ├─ 2291 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --co... ├─12358 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --co... ├─12360 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --co... ├─12362 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --co... [root@prod-openstack-svr ~(keystone_admin)]# neutron agent-list neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +--------------------------------------+----------------------+-----------------------------------+-------------------+-------+----------------+-------------------------------+ | id | agent_type | host | availability_zone | alive | admin_state_up | binary | +--------------------------------------+----------------------+-----------------------------------+-------------------+-------+----------------+-------------------------------+ | 083285ae-3709-4087-934d-38fd674f1bd0 | OVN Metadata agent | prod-openstack-svr.datametica.com | n/a | :-) | True | networking-ovn-metadata-agent | | 3e0fcc51-0016-48f4-823c-20b1d070049d | OVN Controller agent | prod-openstack-svr.datametica.com | n/a | :-) | True | ovn-controller | +--------------------------------------+----------------------+-----------------------------------+-------------------+-------+----------------+-------------------------------+ Regards, Amey. On Fri, 12 Feb 2021 at 22:07, YATIN KAREL <yatinkarel(a)gmail.com&gt; wrote: > > Hi Amey, > > On Fri, Feb 12, 2021 at 9:31 PM Amey Abhyankar <sco1984(a)gmail.com&gt; wrote: >> >> Hello Yatin, >> >> On Thu, 11 Feb 2021 at 14:17, YATIN KAREL <yatinkarel(a)gmail.com&gt; wrote: >> > >> > Hi Amey, >> >> >> Hello, >> >> >> >> >> >> I am trying to install OpenStack PackStack in DELL360 server but getting issues. >> >> >> >> >> >> Details as follows: >> >> >> >> >> >> Hardware: DELL360 PowerEdge >> >> >> Processor: Intel XEON E5-2699 v4 @ 2.20 Ghz [2 physical CPU's] >> >> >> Memory: 512 GB >> >> >> Disk type: Intel SSD's >> >> >> OS tried: Cent OS 7 (2003,2009) , Cent OS 8.3 >> >> >> RDO repo's tried: Victoria,Ussuri,Train >> >snip< >> > Instead of trying all releases, let's stick to one and collect logs to understand the issue, I would suggest you to try packsack default installation with openstack-victoria release, and if it fails >collect logs that i asked in previous mail. >> When there are no proper errors, it is difficult to troubleshoot. >> Finally the issue is resolved. >> It was due to the warning of SMT [1] while installing CentOS. >> I disabled the logical CPU's in BIOS > Installed CentOS 7 > Installed >> packstack Train > All good. >> But now I have half of the CPU threads available for use. >> Very strange issue BTW of SMT + CentOS kernel + PackStack >> >> [1] access.redhat.com/solutions/rhel-smt >> > > > Yeap it's a strange issue, good to hear you get it resolved. /me still didn't get why services were failing to start, No idea how to handle it during deployment, If you can share more details wrt Error, Environment details, how you got to identify that SMT was the issue, etc, maybe you can create a bug so we can see if it's possible to handle the issue in packstack itself if it's packstack related. >> >> > Also you can connect on Freenode IRC channel #rdo where you may get more early responses. >> > >> >> >> Installation type's tried: answer files, advanced networking >> >> > >> >> > Have you also tried out basic packstack installation(sudo packstack --allinone) to ensure it's not related to custom changes in the answer file. >> >> >> >> >> >> Yum update: tried with yum update & without yum update. >> >> > >> >> > Yes it's better to do a yum update before installation to keep the system up to date with latest CentOS packages. >> >> >> >> >> >> >> >> >> I did the same installation on Supermicro server with Intel XEON >> >> >> Silver 4110 @ 2.10 Ghz using the same set of software's & installation >> >> >> methods. No issues found. >> >> >> >> >> > Considering you have not faced issues in other hardware and different releases failed for you on DELL360 PowerEdge seems more like hardware related. >> >> > >> >> >> Anybody encountered similar issues? Thanks. >> >> >> >> >> >> Regards, >> >> >> Amey. >> >> >> _______________________________________________ >> >> >> users mailing list >> >> >> users(a)lists.rdoproject.org >> >> >> http://lists.rdoproject.org/mailman/listinfo/users >> >> >> >> >> >> To unsubscribe: users-unsubscribe(a)lists.rdoproject.org >> >> > >> >> > >> >> > >> >> > Thanks and Regards >> >> > Yatin Karel >> > >> > >> > >> > Thanks and Regards >> > Yatin Karel > > > > Thanks and Regards > Yatin Karel