Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14047

"Too many open files"

    XMLWordPrintable

    Details

      Description

      Using default settings, I got the following error, once the DB reached 250 GB:

      2017-10-10  7:31:56 140711626262272 [ERROR] LibRocksDB:[/opt/slapgrid/7c3e140d2be16dd0fc4001874fc875be/parts/mariadb__compile__/mariadb-10.2.9/storage/rocksdb/rocksdb/db/compaction_job.cc:1240] [default] [JOB 13239] OpenCompactionOutputFiles for table #28903 fails at NewWritableFile with status IO error: While open a file for appending: ./.rocksdb/028903.sst: Too many open files
      2017-10-10  7:31:56 140711626262272 [ERROR] LibRocksDB:[/opt/slapgrid/7c3e140d2be16dd0fc4001874fc875be/parts/mariadb__compile__/mariadb-10.2.9/storage/rocksdb/rocksdb/db/db_impl_compaction_flush.cc:1335] Waiting after background compaction error: IO error: While open a file for appending: ./.rocksdb/028903.sst: Too many open files, Accumulated background error counts: 1
      mysqld: /opt/slapgrid/7c3e140d2be16dd0fc4001874fc875be/parts/mariadb__compile__/mariadb-10.2.9/storage/rocksdb/rocksdb/db/db_impl.cc:710: void rocksdb::DBImpl::MarkLogsSynced(uint64_t, bool, const rocksdb::Status&): Assertion `log.getting_synced' failed.
      171010  7:31:57 [ERROR] mysqld got signal 6 ;
      

      I increased limits so that it can reach 6 TB:

      • rocksdb_max_open_files = 32768
      • rocksdb_merge_buf_size = 256M

      However, RocksDB continues to split the DB in files of 64 MB so I'm not sure I changed the correct variable.

      Anyway, we think that RocksDB should adapt automatically as the DB grows.

      edit: Increasing rocksdb_max_open_files does nothing if open_files_limit is not also increased. Maybe one of them should be calculated automatically from the other.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                psergey Sergei Petrunia
                Reporter:
                jmuchemb Julien Muchembled
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: