Advanced Troubleshooting

Here are some useful commands if you run into problems are want to understand more about the components of the Knowledge Network build pipeline.

1. List all Mesos tasks

curl -L -X GET 127.0.0.1:5050/tasks
curl -L -X GET 127.0.0.1:5050/system/stats.json
curl -L -X GET 127.0.0.1:5050/metrics/snapshot
curl -L -X GET 127.0.0.1:5050/master/slaves

2. List all Marathon job statuses

curl -X GET 127.0.0.1:8080/v2/apps/

3. List all Chronos jobs

curl -L -X GET 127.0.0.1:8888/scheduler/jobs

4. Get statuses of all Chronos job that are not successes

curl -L -s -X GET 127.0.0.1:8888/scheduler/graph/csv | grep node, | \
    awk -F, '{print $3"\t"$4"\t"$1"\t"$2}' | sort | uniq | grep -v success

5. Remove all stopped Docker containers

docker ps -aq --no-trunc | xargs docker rm

6. Get Docker container usage stats

eval "docker inspect --format='{{.Name}}' \$(docker ps -aq --no-trunc) | \
  cut -c 2- | xargs docker stats --no-stream=true"

7. Find Mesos identifiers per pipeline stage

for i in mysqld redis-server check_utilities fetch_utilities table_utilities conv_utilities import_utilities export_utilities KN_starter next_step; do
  echo $i
  docker ps -a --no-trunc | grep $i | rev | cut -d' ' -f 1 | rev | awk -v LABEL="$i" '{print $1"\t"LABEL}'
done;