Issue | Due to unforeseen circumstances (e.g: unexpected shutdown), MongoDB on the NetWitness Admin Server/ESA appliance can become corrupt. The most obvious symptoms of corruption include:
- mongod service failing to start:
[root@nwsvr~]# systemctl status mongod ● mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled) Active: failed (Result: start-limit) since Thu 2019-12-05 03:14:05 UTC; 44s ago Process: 2854 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=14) Process: 2851 ExecStartPre=/bin/bash -c echo never > /sys/kernel/mm/transparent_hugepage/defrag (code=exited, status=0/SUCCESS) Process: 2848 ExecStartPre=/bin/bash -c echo never > /sys/kernel/mm/transparent_hugepage/enabled (code=exited, status=0/SUCCESS) Process: 2845 ExecStartPre=/usr/bin/chown -R mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS) Process: 2843 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS) Main PID: 1729 (code=exited, status=0/SUCCESS)
Dec 05 03:14:05 nwsvr systemd[1]: mongod.service: control process exited, code=exited status=14 Dec 05 03:14:05 nwsvr systemd[1]: Failed to start High-performance, schema-free document-oriented database. Dec 05 03:14:05 nwsvr systemd[1]: Unit mongod.service entered failed state. Dec 05 03:14:05 nwsvr systemd[1]: mongod.service failed. Dec 05 03:14:05 nwsvr systemd[1]: mongod.service holdoff time over, scheduling restart. Dec 05 03:14:05 nwsvr systemd[1]: start request repeated too quickly for mongod.service Dec 05 03:14:05 nwsvr systemd[1]: Failed to start High-performance, schema-free document-oriented database. Dec 05 03:14:05 nwsvr systemd[1]: Unit mongod.service entered failed state. Dec 05 03:14:05 nwsvr systemd[1]: mongod.service failed.
AND: - /var/log/mongodb/mongod.log being flooded with repeated messages similar to these:
2019-12-05T04:07:54.321+0000 I CONTROL [main] ***** SERVER RESTARTED ***** 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] MongoDB starting : pid=17354 port=27017 dbpath=/var/netwitness/mongo 64-bit host=nwsvr 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] db version v3.6.4 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] git version: d0181a711f7e7f39e60b5aeb1dc7097bf6ae5856 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] allocator: tcmalloc 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] modules: none 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] build environment: 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] distmod: rhel70 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] distarch: x86_64 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] target_arch: x86_64 2019-12-05T04:07:54.404+0000 I CONTROL [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "0.0.0.0", port: 27017, ssl: { CAFile: "/etc/pki/nw/trust/truststore.pem", PEMKeyFile: "/etc/pki/nw/mongo/mongod-combined.pem", allowConnectionsWithoutCertificates: true, allowInvalidCertificates: false, allowInvalidHostnames: true, disabledProtocols: "TLS1_0,TLS1_1", mode: "preferSSL", sslCipherConfig: "HIGH:!EXPORT:!aNULL@STRENGTH:!kRSA" } }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, security: { authorization: "enabled" }, setParameter: { internalQueryExecMaxBlockingSortBytes: "134217728", opensslDiffieHellmanParameters: "/etc/pki/nw/mongo/dhparam2048.pem" }, storage: { dbPath: "/var/netwitness/mongo", journal: { enabled: true }, wiredTiger: { engineConfig: { cacheSizeGB: 16.0 } } }, systemLog: { destination: "file", logAppend: true, logRotate: "reopen", path: "/var/log/mongodb/mongod.log" } } 2019-12-05T04:07:54.405+0000 I - [initandlisten] Detected data files in /var/netwitness/mongo created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. 2019-12-05T04:07:54.405+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=16384M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),cache_cursors=false,log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress), 2019-12-05T04:07:55.052+0000 E STORAGE [initandlisten] WiredTiger error (0) [1575518875:52721][17354:0x7f3478cfab00], file:WiredTiger.wt, WT_CURSOR.insert: read checksum error for 20480B block at offset 942080: block header checksum of 0 doesn't match expected checksum of 662513522 2019-12-05T04:07:55.052+0000 E STORAGE [initandlisten] WiredTiger error (0) [1575518875:52886][17354:0x7f3478cfab00], file:WiredTiger.wt, WT_CURSOR.insert: {942080, 20480, 662513522}: (chunk 1 of 20): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <snipped> 2019-12-05T04:07:55.056+0000 E STORAGE [initandlisten] WiredTiger error (0) [1575518875:56262][17354:0x7f3478cfab00], file:WiredTiger.wt, WT_CURSOR.insert: {942080, 20480, 662513522}: (chunk 20 of 20): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2019-12-05T04:07:55.056+0000 E STORAGE [initandlisten] WiredTiger error (0) [1575518875:56485][17354:0x7f3478cfab00], file:WiredTiger.wt, WT_CURSOR.insert: WiredTiger.wt: encountered an illegal file format or internal value: (__wt_block_read_off, 302) 2019-12-05T04:07:55.056+0000 E STORAGE [initandlisten] WiredTiger error (-31804) [1575518875:56504][17354:0x7f3478cfab00], file:WiredTiger.wt, WT_CURSOR.insert: the process must exit and restart: WT_PANIC: WiredTiger library panic 2019-12-05T04:07:55.056+0000 F - [initandlisten] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361 2019-12-05T04:07:55.056+0000 F - [initandlisten]
***aborting after fassert() failure
2019-12-05T04:07:55.149+0000 F - [initandlisten] Got signal: 6 (Aborted).
0x55b88862a861 0x55b888629a79 0x55b888629f5d 0x7f34777766d0 0x7f34773d0277 0x7f34773d1968 0x55b886d8b425 0x55b886e582ce 0x55b886ec4ba1 0x55b886d2827e 0x55b886d285a7 0x55b886f73997 0x55b886f73ad5 0x55b886eec7a3 0x55b886ef2e71 0x55b886f160a6 0x55b886f861b7 0x55b886f2f006 0x55b886edbf9a 0x55b886f53ae4 0x55b886edcf77 0x55b886e6c597 0x55b886e6923c 0x55b886e3b95e 0x55b886e1ff04 0x55b887014b67 0x55b886d240e7 0x55b886dff18c 0x55b886d8d0e9 0x7f34773bc445 0x55b886deea9f ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"55B8863FC000","o":"222E861","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55B8863FC000","o":"222DA79"},{"b":"55B8863FC000","o":"222DF5D"},{"b":"7F3477767000","o":"F6D0"},{"b":"7F347739A000","o":"36277","s":"gsignal"},{"b":"7F347739A000","o":"37968","s":"abort"},{"b":"55B8863FC000","o":"98F425","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"55B8863FC000","o":"A5C2CE"},{"b":"55B8863FC000","o":"AC8BA1"},{"b":"55B8863FC000","o":"92C27E","s":"__wt_err"},{"b":"55B8863FC000","o":"92C5A7","s":"__wt_panic"},{"b":"55B8863FC000","o":"B77997","s":"__wt_block_read_off"},{"b":"55B8863FC000","o":"B77AD5","s":"__wt_bm_read"},{"b":"55B8863FC000","o":"AF07A3","s":"__wt_bt_read"},{"b":"55B8863FC000","o":"AF6E71","s":"__wt_page_in_func"},{"b":"55B8863FC000","o":"B1A0A6","s":"__wt_row_search"},{"b":"55B8863FC000","o":"B8A1B7","s":"__wt_btcur_insert"},{"b":"55B8863FC000","o":"B33006"},{"b":"55B8863FC000","o":"ADFF9A"},{"b":"55B8863FC000","o":"B57AE4","s":"__wt_log_scan"},{"b":"55B8863FC000","o":"AE0F77","s":"__wt_txn_recover"},{"b":"55B8863FC000","o":"A70597","s":"__wt_connection_workers"},{"b":"55B8863FC000","o":"A6D23C","s":"wiredtiger_open"},{"b":"55B8863FC000","o":"A3F95E","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"55B8863FC000","o":"A23F04"},{"b":"55B8863FC000","o":"C18B67","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"55B8863FC000","o":"9280E7"},{"b":"55B8863FC000","o":"A0318C","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},{"b":"55B8863FC000","o":"9910E9","s":"main"},{"b":"7F347739A000","o":"22445","s":"__libc_start_main"},{"b":"55B8863FC000","o":"9F2A9F"}],"processInfo":{ "mongodbVersion" : "3.6.4", "gitVersion" : "d0181a711f7e7f39e60b5aeb1dc7097bf6ae5856", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-862.11.6.el7.x86_64", "version" : "#1 SMP Tue Aug 14 21:49:04 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "55B8863FC000", "elfType" : 3, "buildId" : "37115F864F27162060BAC158CC814CBD48741B71" }, { "b" : "7FFC4ED60000", "elfType" : 3, "buildId" : "2562E90AE9BCFB0D02589A7C3B20B07AC58D6A74" }, { "b" : "7F34788D5000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "2BDC2B6FF0B2C204CCE34D139A9EADA0272EB070" }, { "b" : "7F3478662000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "9112A44B60B06C5FE236C7B1EF38A8D74C16AC0C" }, { "b" : "7F34782A7000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "C4F0A80793B40F2E40596C49987D21EE7B9C4742" }, { "b" : "7F347809F000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "D33989EC31EFE745EB0D3B68A92D19E77D7DDFDA" }, { "b" : "7F3477E9B000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "5CDB5A56336E7E2BD14FFA189411E44A834AFCD8" }, { "b" : "7F3477B99000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "F4CAE74047F9AA2D5A71FDEC67C4285D75753EBA" }, { "b" : "7F3477983000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "531AA1391EA4E1489D5EF11AA5DC2FFD9E2BDFEE" }, { "b" : "7F3477767000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "F4C04BCE85D2D269D0A2AF4972FC69805B50345B" }, { "b" : "7F347739A000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "CB4B7554D1ADBEF2F001142DD6F0A5139FC9AA69" }, { "b" : "7F3478AEE000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D266B1F6650927E18108323BCCA8F7B68E68EB92" }, { "b" : "7F347714D000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "16FE0DC6CEFC5F444BC876516D02EFE9CC2D432F" }, { "b" : "7F3476E65000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "D1CD1B94855A85FBC735C745DB39BC096F7D8CC3" }, { "b" : "7F3476C61000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "A3832734347DCA522438308C9F08F45524C65C9B" }, { "b" : "7F3476A2E000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "A20F715C514B3EA873F4CC77D585A50CB670E266" }, { "b" : "7F3476818000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EA8E45DC8E395CC5E26890470112D97A1F1E0B65" }, { "b" : "7F34765C7000", "path" : "/lib64/libbwrap.so.1", "elfType" : 3, "buildId" : "C8B57E5EDB72A0EB6BF8E4ABFD978CCC0CF7A12A" }, { "b" : "7F34763B9000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "9F82B06CE44724A8ACE827A2C95A9A76619EA314" }, { "b" : "7F34761B5000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F3475F8E000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "A88379F56A51950A33198890D37F5F8AEE71F8B4" }, { "b" : "7F3475D2C000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }, { "b" : "7F3475ACE000", "path" : "/usr/lib64/bwrap-1.2.2.2/libcryptocme.so", "elfType" : 3, "buildId" : "2CDC228D1BF20A18FB4BC1BCD9ECBE8F6212AC9F" }, { "b" : "7F34758CA000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_error_info.so", "elfType" : 3, "buildId" : "848E659A077F1D14A8AD96533D18A2BF2E012877" }, { "b" : "7F34756C6000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_aux_entropy.so", "elfType" : 3, "buildId" : "FF51846DA67E57C6BC3BACF69B084C9D45DF5CD4" }, { "b" : "7F347542E000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_base.so", "elfType" : 3, "buildId" : "A31E5C5B44F05A120D258AA19A59658E4EBCDE7D" }, { "b" : "7F34751D0000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_asym.so", "elfType" : 3, "buildId" : "F44DA44D4B1515200C589C0A77DFBD2E76668BC0" }, { "b" : "7F3474F0B000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_ecc_accel_fips.so", "elfType" : 3, "buildId" : "3980C59BA73A4E76F4CD807578C600E91D2D4F18" }, { "b" : "7F3474C6C000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_ecc.so", "elfType" : 3, "buildId" : "7365F5D8E99D2A82BF26EE72B63C80137FC03D9D" }, { "b" : "7F34749DE000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_base_non_fips.so", "elfType" : 3, "buildId" : "65ADC3FB0F2E7E7733198E983BA78215440DACA1" }, { "b" : "7F3474721000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_ecc_accel_non_fips.so", "elfType" : 3, "buildId" : "50EEBC9B0FF5EE82BA77E82895F1AD474A21D490" }, { "b" : "7F34744CA000", "path" : "/usr/lib64/bwrap-1.2.2.2/libccme_ecc_non_fips.so", "elfType" : 3, "buildId" : "439D6B449A34920838E74828E88D420CBC1BAC3E" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55b88862a861] mongod(+0x222DA79) [0x55b888629a79] mongod(+0x222DF5D) [0x55b888629f5d] libpthread.so.0(+0xF6D0) [0x7f34777766d0] libc.so.6(gsignal+0x37) [0x7f34773d0277] libc.so.6(abort+0x148) [0x7f34773d1968] mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55b886d8b425] mongod(+0xA5C2CE) [0x55b886e582ce] mongod(+0xAC8BA1) [0x55b886ec4ba1] mongod(__wt_err+0x9D) [0x55b886d2827e] mongod(__wt_panic+0x33) [0x55b886d285a7] mongod(__wt_block_read_off+0x547) [0x55b886f73997] mongod(__wt_bm_read+0x135) [0x55b886f73ad5] mongod(__wt_bt_read+0x203) [0x55b886eec7a3] mongod(__wt_page_in_func+0x19F1) [0x55b886ef2e71] mongod(__wt_row_search+0x856) [0x55b886f160a6] mongod(__wt_btcur_insert+0x1027) [0x55b886f861b7] mongod(+0xB33006) [0x55b886f2f006] mongod(+0xADFF9A) [0x55b886edbf9a] mongod(__wt_log_scan+0xCE4) [0x55b886f53ae4] mongod(__wt_txn_recover+0x417) [0x55b886edcf77] mongod(__wt_connection_workers+0x37) [0x55b886e6c597] mongod(wiredtiger_open+0x196C) [0x55b886e6923c] mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x89E) [0x55b886e3b95e] mongod(+0xA23F04) [0x55b886e1ff04] mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x637) [0x55b887014b67] mongod(+0x9280E7) [0x55b886d240e7] mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0x86C) [0x55b886dff18c] mongod(main+0x9) [0x55b886d8d0e9] libc.so.6(__libc_start_main+0xF5) [0x7f34773bc445] mongod(+0x9F2A9F) [0x55b886deea9f] ----- END BACKTRACE -----
|