Get help with Cisco Meeting Server and Cisco Meeting Apps

Find answers quickly with these FAQs.



» »

What common syslog messages might I see, and what do they mean?

Cisco Meeting Server encompasses several internal components, for example, media, database, XMPP, WebRTC, recorder/streamer. It’s not expected that an admin be able to troubleshoot issues on their own, but having the basic knowledge to interpret syslog messages and their meanings will be the first step in any troubleshooting task.

Note: this is a list of some common syslog messages and not an exhaustive list of all syslog messages. This is an evolving list and will be added to in the future.

Note: For simplicity, this FAQ does not correlate any Cisco Meeting Server versions with any syslog messages.  Also, to make the messages more readable in the examples below, information such as timestamps, components, server name that normally precedes the actual message is removed.  

The messages are categorized into specific features relating to the Meeting Server internal components.

WebRTC

Syslog messages relating to WebRTC connection is usually due to authentication failure or trust connection failure.

Message:

webbridge-reverse-proxy[1]:  2019/03/27 05:40:43 (WARN):authentication failed for 192.168.6.25:33152: Certificate not in whitelist

Meaning:

Found in the web bridge syslog and warns the user that authentication has failed due to certificate verification failure. In most case, this means that the callbridge certificate is not in the web bridge cert trust store.

 

Message:

INFO : [SESSIONS] Created session: 8b4815cc-f7e6-4903-9a92-ddac329244c0

INFO : XMPP connection dropped while session was live for reason 2

INFO : No DNS SRV records for _xmpp-client._tcp.example.com

WARNING : [SESSIONS] Session acanouser1@example.com: XMPP session disconnected: DNS error

 

Meaning:

Web bridge fails to reach to XMPP server due to DNS SRV record issue. Failure to create DNS A record for XMPP host server is likely to be the cause.

 

Message:

INFO : failed to establish connection to web bridge link 1 (connection failure)

Meaning:

Call bridge cannot establish https connection to the web bridge. It may be due to web bridge not trusting the call bridge or call bridge certificate not in the web bridge trust store.

 

Message:

INFO : LDAP authorisation failed for user 'acanouser1@example.com'

INFO : LDAP failure 49 (invalidCredentials) server diagnostics message: 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1

WARNING : authentication request from acanouser1@example.com took 2 seconds (start: 0, end: 2065)

INFO : unsuccessful login request from acanouser1@example.com

Meaning:

When “Sign In” is used in the WebRTC browser application, LDAP authentication fails due to wrong password provided.

 

Message:

INFO : no user 'acanouser11@example.com' found for authorisation

INFO :unsuccessful login request from acanouser11@example.com

Meaning:

When “Sign In” is used in WebRTC browser application, LDAP authentication fails due to wrong user name provided.

 

Message:

E-MLink-Error TLS 140543964186432  error  140943FC  SSL routines    SSL3_READ_BYTES  sslv3 alert bad record mac s3_pkt.c  1300 SSL alert number 20

Meaning:

The XMPP server generates this message when the TLS XMPP connection disconnects. Despite sounding serious, this message does not necessarily point to something bad going on because there are valid / expected reasons why the connection would no longer be active.

 

Database Operations

The following messages are found on a clustered database deployment.

 

Message:

Health check 172.16.5.80: error (up = 1): operation timed out

Meaning:

One of the nodes lost connection to database peer in database clustering setup.

 

Message:

FATAL:  could not connect to the primary server: could not connect to server: No route to host

Meaning:

The slave database node lost connection to the master database.

 

Message:

LOG:  shutting down

LOG:  database system is shut down

Meaning:

The database server is separated from the database cluster.

 

Message:

ERROR:  Database connection disconnected

Meaning:

The call bridge doesn't have a usable connection to the database. On startup, this is usually because the call bridge is still querying the database to determine the schema version – until this is complete, the call bridge won't consider that database usable.

 

Note that "ERROR : Database connection disconnected" and "INFO : Database connection now OK" are generated at the start and end of this non-reachable condition, and correspond to the start and end of a call bridge alarm.

 

Streaming

Meeting Server supports sending of media over RTMP to a configured URL while the meeting is active.

These messages are found on the call bridge syslog:

Message:

streaming device 3: available (0 streamings)

Meaning:

The streamer is available. The value within the parentheses gives number of streams.

 

Message:

streaming device 3: unavailable (3)

Meaning:

The streamer is unavailable.

 

Message:

stopping streaming (space 'bob.jones's cospace')

Meaning:

The space stops being streamed.

 

Message:

"unable to start streaming (space 'bob.jones cospace') -- streaming limit reached"

Meaning:

The stream limit has been reached, possibly due to insufficient recording license to stream to space.

 

 

The messages below are found on the streamer syslog:

 

Message:

streamer.1e5a2877-6fef-43cb-8a4e-fbd42a17bc2c[33]: Connecting to 'a.rtmp.youtube.com', app 'live2', stream 'asdf-1234-asdf-1234', port '1935', scheme 'rtmp

Meaning:

This indicates the streamer is connecting to the streaming server.

 

Message:

Connecting to RTMP server failed (Timeout)

Meaning:

Indicates connection timing out due to a variety of reasons. Logs from the streaming server, as well as packet capture on the streamer, may be useful for determining the cause.

 

 

Message:

Initiating RTMP protocol failed (connection closed by far end)

Meaning:

The streaming server closes the connection on the streamer during the initiation. Logs from the streaming server, as well as packet capture on the streamer, may be useful for determining the cause.

 

Message:

"RTMP stream url has a bad format"

Meaning:

The stream URL is in the wrong format.

 

Message:

"Connecting to RTMP server failed ([Errno -2] Name or service not known)"

Meaning:

Streamer is unable to connect to RTMP server, possibly due to streamUrl not starting with 'rtmp://' or the hostname isn't resolvable.

 

Office365 Support

Message:

lync simplejoin resolution: resolution failed (invalid simplejoin url)

Meaning:

The simplejoin URL contained invalid characters, thus DNS lookup was not attempted.

 

Message:

lync simplejoin resolution: resolution failed (connection failure)

Meaning:

Call bridge fails to make HTTPS connection to O365. Packet capture may be helpful to determine where in the HTTPS connectionto the Lync server is unsuccessful.

 

Message:

lync conference resolution: conference "meet.ync.com/silverflare/bob.jones/6scknp9h" not found

Meaning:

Call bridge performing DNS lookup of meet.ync.com failed. DNS tracing should be enabled to show more details.

 

Load Balancing Calls

Meeting Server supports load balancing calls where calls can be hosted on a different call bridge from which the calls first originally connected to.

Message:

replace query for conference 428e8238-715b-4df1-9d9f-f628a3770763: response from 'CB1_US' (priority: 0, load level: 0, conference is running: 1)

Meaning:

This call bridge queries another call bridge 'CB1_US' to see if that call bridge hosts this meeting id 428e8238-715b-4df1-9d9f-f628a3770763. Then the response from CB1_US indicates that there is already 1 active conference on that call bridge.

 

Message:

replace query for conference 428e8238-715b-4df1-9d9f-f628a3770763: using remote server 'CB2_US' (priority: 0, load level: 0, conference is running: 1)

Meaning:

This call bridge queries another call bridge 'CB1_US' to see if that call bridge hosts this meeting id 428e8238-715b-4df1-9d9f-f628a3770763. Then the response from CB1_US indicates that there is already 1 active conference on that call bridge.

 

Message:

replace query for conference 428e8238-715b-4df1-9d9f-f628a3770763: using remote server 'CB2_US' (priority: 0, load level: 0, conference is running: 1)

Meaning:

This call bridge is using a remote call callbridge 'CB2_US' to host meeting id 428e8238-715b-4df1-9d9f-f628a3770763. Even though the call landed on this call bridge, the incoming call is hosted on another call bridge 'CB2_US'.

 

Message:

replace query for conference 428e8238-715b-4df1-9d9f-f628a3770763: using local server 'CB1_US' (priority: 0, load level: 0, conference is running: 1)

Meaning:

This call bridge is using itself as the callbridge to host meeting id 428e8238-715b-4df1-9d9f-f628a3770763.

 

Message:

replacing call '89373ZDI4NWY3MGM1ZmMzMDJkMWQ0MjRlOTMzODIzZjE0MzY' to conference 428e8238-715b-4df1-9d9f-f628a3770763 on server 'cb1.acompany.com'

Meaning:

On the Meeting Server that received the call, it is replacing this call to conference 428e8238-715b-4df1-9d9f-f628a3770763 on another remote server 'cb1.acompany.com'.

 

Message:

replacing call '89373YmNiMWM4OWI5ZGRiOTUyMWQ4NzFlMGEyNWNhZGRhYjI' from server 'cb1.acompany.com' into conference 428e8238-715b-4df1-9d9f-f628a3770763

Meaning:

On the Meeting Server hosting the call, this indicates call for conference 428e8238-715b-4df1-9d9f-f628a3770763 came from another server 'cb1.acompany.com'.

 

Vbrick Rev Integration

From version 2.3, Meeting Server provides upload directly to Vbrick Rev for video viewing and editing.

Message:

while processing video: 20170921185115-0400.mp4 saw exception:
    HTTPSConnectionPool(host='acanoserver10.acanodemo.com', port=443): Max retries exceeded with url: /api/v1/coSpaces/2c16e0b6-821c-4bcf-8785-3b80de7471fc (Caused by
    NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f008a5882d0>: Failed to establish a new connection: [Errno 111]

Meaning:

Connection is broken between webadmin and Vbrick server.

 

Message:

ConnectionError: HTTPSConnectionPool(host='acanoserver10.acanodemo.com', port=443): Max retries exceeded with url:
    /api/v1/coSpaces/2c16e0b6-821c-4bcf-8785-3b80de7471fc (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f55252b8950>: Failed to
    establish a new connection: [Errno 111] Connection refused',))

Meaning:

There is a mismatch between the webadmin port defined on the Meeting Server call bridge and the 'cms port' configured on Vbrick server.

 

Message:

while processing video: 20170921190857-0400.mp4  saw exception: HTTPSConnectionPool(host='bobobob.acanodemo.com',
    port=443): Max retries exceeded with url: /api/v1/coSpaces/2c16e0b6-821c-4bcf-8785-3b80de7471fc (Caused by
    NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f2481c2d210>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Meaning:

Vbrick Meeting Server host is not valid or fail DNS resolution. To check, perform DNS lookup on this server to see if the name is returned. Also check DNS configuration on the Vbrick server to confirm that the server is properly configured.

 

Message:

SSLError: hostname '192.168.11.43' doesn't match either of 'acanoserver10.acanodemo.com',   'join.acanoserver10.acanodemo.com'

Meaning:

There is a mismatch between the ip address/hostname configured under Vbrick Meeting Server host and the hostname received from the web admin certificate.

 

Message:

HTTPError: 401 Client Error: Unauthorized for url: https://rev:443/api/v1/user/login

Meaning:

Username for the Vbrick component is incorrect.

 

Message:

HTTPError: 401 Client Error: Unauthorized for url: https://cmsserver:5443/api/v1/coSpaces/xxxxx-yyyyy-zzzzzz

Meaning:

While communicating with Meeting Server web admin, Vbrick component generates an error when username and password are incorrect.

 

Message:

while processing video: 20170929104821-0400.mp4  saw exception: [Errno 30] Read-only file system

Meaning:

Vbrick component is unable to write to NFS Share possibly due to permission. Verify that READ and WRITE permission is set on NFS Share.

 

Unified CM Adhoc Call Escalation

This feature enables a point-to-point call to escalate to a multipoint call via Unified CM adhoc call escalation.

Message:

[WARNING] cucm-esc.CodianToCms - Received "401" response creating space "bob.jones.cospace":#012{}

Meaning:

On Unified CM ad-hoc escalation call, Meeting Server attempted to create cospace via API using provided credentials and received error '401' . The failure to create the space is due to incorrect username/password.

 

System Messages

These messages are not related to a specific feature but for the internal operation of Meeting Server.

Message:

user.warning mmp syslog: log file size 26214435 bytes exceeds 75% of capacity

Meaning:

The same warning could also apply to an audit file. This occurs when the available storage of the remote server reaches 75% of its disk queue (either total disk space or space allocated for the folder).