• 更新日志
最后更新于
Introduced .
and searches during updates are no longer blocked by chunk merging.
⚠️ BREAKING Introduced with new options: , for create table / alter table.
Improved clarity in chunk merging.
Added support for .
Implemented secondary indexes for / functions.
HTTP requests now support Content-Encoding: gzip.
Allowed Buddy request to daemon to bypass the constraint.
Updated default to 128MB.
Added support for in JSON .
Enhanced writing synchronization to prevent errors.
msc scale for aggs at compat endpoints.
Implemented autodetection of data types imported from Elasticsearch.
⚠️ BREAKING Incorporated zlib support in Windows packages.
⚠️ BREAKING now compatible with distributed tables.
Released: Oct 7 2024
Released: August 2nd 2024
Version 6.3.6 continues the 6.3 series and includes only bug fixes.
Released: July 31st 2024
Version 6.3.4 continues the 6.3 series and includes only minor improvements and bug fixes.
Released: June 26th 2024
Version 6.3.2 continues the 6.3 series and includes several bug fixes, some of which were discovered after the release of 6.3.0.
Released: May 23rd 2024
First, cleanly stop all your nodes
Then, start the node that was stopped last with --new-cluster, using the tool manticore_new_cluster in Linux.
Released: August 23rd 2023
Version 6.2.12 continues the 6.2 series and addresses issues discovered after the release of 6.2.0.
Released: August 4th 2023
The query optimizer has been enhanced to support full-text queries, significantly improving search efficiency and performance.
Integrations with:
Significant performance improvement in count distinct operation by employing a combination of hash tables and HyperLogLog.
Enabled multithreaded execution of queries containing secondary indexes, with the number of threads limited to the count of physical CPU cores. This should considerably improve the query execution speed.
pseudo_sharding has been adjusted to be limited to the number of free threads. This update considerably enhances the throughput performance.
Field and attribute order is now consistent between SHOW CREATE TABLE and DESC operations.
If disk space is insufficient when executing INSERT queries, new INSERT queries will fail until enough disk space becomes available.
When count(*) is used with a single filter, queries now leverage precalculated data from secondary indexes when available, substantially speeding up query times.
⚠️ Tables created or modified in version 6.2.0 cannot be read by older versions.
⚠️ Document IDs are now handled as unsigned 64-bit integers during indexing and INSERT operations.
⚠️ The syntax for query optimizer hints has been updated. The new format is /*+ SecondaryIndex(uid) */. Please note that the old syntax is no longer supported.
⚠️ String fields/attributes marked as indexed and attribute are now regarded as a single field during INSERT, DESC, and ALTER operations.
Released: March 15 2023
Improved integration with Logstash, Beats etc. including:
Support for Logstash versions 7.6 - 7.15, Filebeat versions 7.7 - 7.12
Auto-schema support.
Added handling of bulk requests in Elasticsearch-like format.
Released: Feb 10 2023
Released: Feb 7 2023
This release also includes more than 130 bug fixes and numerous features, many of which can be considered major.
when you migrate from Manticore 5 to the newer version,
Dynamic max_matches for aggregation queries to increase accuracy and lower response time.
As we recently added support for secondary indexes, things became confusing as "index" could refer to a secondary index, a full-text index, or a plain/real-time index. To reduce confusion, we are renaming the latter to "table". The following SQL/command line commands are affected by this change. Their old versions are deprecated, but still functional:
index <table name> => table <table name>,
searchd -i / --index => searchd -t / --table,
SHOW INDEX STATUS => SHOW TABLE STATUS,
SHOW INDEX SETTINGS => SHOW TABLE SETTINGS,
FLUSH RTINDEX => FLUSH TABLE,
OPTIMIZE INDEX => OPTIMIZE TABLE,
ATTACH TABLE plain TO RTINDEX rt => ATTACH TABLE plain TO TABLE rt,
RELOAD INDEX => RELOAD TABLE,
RELOAD INDEXES => RELOAD TABLES.
We are not planning to make the old forms obsolete, but to ensure compatibility with the documentation, we recommend changing the names in your application. What will be changed in a future release is the "index" to "table" rename in the output of various SQL and JSON commands.
Queries with stateful UDFs are now forced to be executed in a single thread.
⚠️ BREAKING CHANGE: Columnar storage format has been changed. You need to rebuild those tables that have columnar attributes.
⚠️ BREAKING CHANGE: Secondary indexes file format has been changed, so if you are using secondary indexes for searching and have searchd.secondary_indexes = 1 in your configuration file, be aware that the new Manticore version will skip loading the tables that have secondary indexes. It's recommended to:
Before you upgrade change searchd.secondary_indexes to 0 in the configuration file.
Run the instance. Manticore will load up the tables with a warning.
Run ALTER TABLE <table name> REBUILD SECONDARY for each index to rebuild secondary indexes.
⚠️ BREAKING CHANGE: The binlog version has been updated, so any binlogs from previous versions will not be replayed. It is important to ensure that Manticore Search is stopped cleanly during the upgrade process. This means that there should be no binlog files in /var/lib/manticore/binlog/ except for binlog.meta after stopping the previous instance.
⚠️ BREAKING CHANGE: Replication protocol has been changed. If you are running a replication cluster, then when upgrading to Manticore 5 you need to:
stop all your nodes first cleanly
and then start the node which was stopped last with --new-cluster (run tool manticore_new_cluster in Linux).
Refactoring of Secondary indexes integration with Columnar storage.
RHEL 9 support including Centos 9, Alma Linux 9 and Oracle Linux 9.
It's now possible to install a specific version using APT.
Switched to compiling using CLang 15.
⚠️ BREAKING CHANGE: Custom Homebrew formulas including the formula for Manticore Columnar Library. To install Manticore, MCL and any other necessary components, use the following command brew install manticoresoftware/manticore/manticoresearch manticoresoftware/manticore/manticore-extra.
Released: May 30th 2022
Released: May 18th 2022
Faster bulk INSERT/REPLACE/DELETE via JSON over HTTP: previously you could provide multiple write commands via HTTP JSON protocol, but they were processed one by one, now they are handled as a single transaction.
reduces peak RAM usage, lowering the risk of OOM
decreases response time (our tests indicated an 11% reduction for processing a 100MB batch)
Previously (note the response time):
Now:
Having at least one full-text field in a real-time/plain index is not mandatory anymore. You can now use Manticore even in cases not having anything to do with full-text search.
⚠️ BREAKING CHANGE: Index meta file format change. Previously meta files (.meta, .sph) were in binary format, now it's just json. The new Manticore version will convert older indexes automatically, but:
you can get warning like WARNING: ... syntax error, unexpected TOK_IDENT
you won't be able to run the index with previous Manticore versions, make sure you have a backup
Faster replication SST
⚠️ BREAKING CHANGE: Replication protocol has been changed. If you are running a replication cluster, then when upgrading to Manticore 5 you need to:
stop all your nodes first cleanly
and then start the node which was stopped last with --new-cluster (run tool manticore_new_cluster in Linux).
Replication improvements:
Faster SST
Noise resistance which can help in case of unstable network between replication nodes
Improved logging
Security improvement: Manticore now listens on 127.0.0.1 instead of 0.0.0.0 in case no listen at all is specified in config. Even though in the default configuration which is shipped with Manticore Search the listen setting is specified and it's not typical to have a configuration with no listen at all, it's still possible. Previously Manticore would listen on 0.0.0.0 which is not secure, now it listens on 127.0.0.1 which is usually not exposed to the Internet.
Faster aggregation over columnar attributes.
Increased AVG() accuracy: previously Manticore used float internally for aggregations, now it uses double which increases the accuracy significantly.
Improved support for JDBC MySQL driver.
Significant memory consumption improvements: Manticore consumes significantly less RAM now in case of long and intensive insert/replace/optimize workload in case stored fields are used.
⚠️ Format change of the response of /bulk INSERT/REPLACE/DELETE requests:
previously each sub-query constituted a separate transaction and resulted in a separate response
now the whole batch is considered a single transaction, which returns a single response
⚠️ Search options low_priority and boolean_simplify now require a value (0/1): previously you could do SELECT ... OPTION low_priority, boolean_simplify, now you need to do SELECT ... OPTION low_priority=1, boolean_simplify=1.
⚠️ HTTP JSON requests are now logged in different format in mode query_log_format=sphinxql. Previously only full-text part was logged, now it's logged as is.
⚠️ BREAKING CHANGE: because of the new structure when you upgrade to Manticore 5 it's recommended to remove old packages before you install the new ones:
RPM-based: yum remove manticore*
Debian and Ubuntu: apt remove manticore*
New deb/rpm packages structure. Previous versions provided:
manticore-server with searchd (main search daemon) and all needed for it
manticore-tools with indexer and indextool
manticore including everything
manticore-all RPM as a meta package referring to manticore-server and manticore-tools
The new structure is:
manticore - deb/rpm meta package which installs all the above as dependencies
manticore-server-core - searchd and everything to run it alone
manticore-server - systemd files and other supplementary scripts
manticore-tools - indexer, indextool and other tools
manticore-common - default configuration file, default data directory, default stopwords
manticore-icudata, manticore-dev, manticore-converter didn't change much
.tgz bundle which includes all the packages
Support for Ubuntu Jammy
PQ transactions are now atomic and isolated. Previously PQ transactions support was limited. It enables much faster REPLACE into PQ, especially when you need to replace a lot of rules at once. Performance details:
4.0.2
It takes 48 seconds to insert 1M PQ rules and 406 seconds to REPLACE just 40K in 10K batches.
4.2.0
It takes 34 seconds to insert 1M PQ rules and 23 seconds to REPLACE them in 10K batches.
Binlog version was increased, binlog from previous version won't be replayed, so make sure you stop Manticore Search cleanly during upgrade: no binlog files should be in /var/lib/manticore/binlog/ except binlog.meta after stopping the previous instance.
searchd.workers was obsoleted since 3.5.0, now it's deprecated, if you still have it in your configuration file it will trigger a warning on start. Manticore Search will start, but with a warning.
If you use PHP and PDO to access Manticore you need to do PDO::ATTR_EMULATE_PREPARES
in real-time indexes for INSERT, REPLACE, DELETE, OPTIMIZE
in replication
in ALTER
in indextool --check
Chunk snapshots and locks system revamp. These changes may be invisible from outside at first glance, but they improve the behaviour of many things happening in real-time indexes significantly. In a nutshell, previously most Manticore data manipulation operations relied on locks heavily, now we use disk chunk snapshots instead.
read operations (e.g. SELECTs, replication) are performed with snapshots
operations that just change internal index structure without modifying schema/documents (e.g. merging RAM segments, saving disk chunks, merging disk chunks) are performed with read-only snapshots and replace the existing chunks in the end
UPDATEs and DELETEs are performed against existing chunks, but for the case of merging that may be happening the writes are collected and are then applied against the new chunks
UPDATEs acquire an exclusive lock sequentially for every chunk. Merges acquire a shared lock when entering the stage of collecting attributes from the chunk. So at the same time only one (merge or update) operation has access to attributes of the chunk.
when merging gets to the phase, when it needs attributes it sets a special flag. When UPDATE finishes, it checks the flag, and if it's set, the whole update is stored in a special collection. Finally, when the merge finishes, it applies the updates set to the newborn disk chunk.
ALTER runs via an exclusive lock
replication runs as a usual read operation, but in addition saves the attributes before SST and forbids updates during the SST
faster update by id via HTTP in big indexes in some cases (depends on the ids distribution)
3.6.0
4.0.2
the new version can read older indexes, but the older versions can't read Manticore 4's indexes
removed implicit sorting by id. Sort explicitly if required
charset_table's default value changes from 0..9, A..Z->a..z, _, a..z, U+410..U+42F->U+430..U+44F, U+430..U+44F, U+401->U+451, U+451 to non_cjk
OPTIMIZE happens automatically. If you don't need it make sure to set auto_optimize=0 in section searchd in the configuration file
for contributors: we now use Clang compiler for Linux builds as according to our tests it can build a faster Manticore Search and Manticore Columnar Library
make sure you a stop Manticore 3 cleanly:
no binlog files should be in /var/lib/manticore/binlog/ (only binlog.meta should be in the directory)
otherwise the indexes Manticore 4 can't reply binlogs for won't be run
the new version can read older indexes, but the older versions can't read Manticore 4's indexes, so make sure you make a backup if you want to be able to rollback the new version easily
if you run a replication cluster make sure you:
stop all your nodes first cleanly
and then start the node which was stopped last with --new-cluster (run tool manticore_new_cluster in Linux).
Lots of replication issues have been fixed:
Maintenance release before Manticore 4
Fully revised histograms. When building an index Manticore also builds histograms for each field in it, which it then uses for faster filtering. In 3.6.0 the algorithm was fully revised and you can get a higher performance if you have a lot of data and do a lot of filtering.
tool manticore_new_cluster [--force] useful for restarting a replication cluster via systemd
chunk id for a merged disk chunk is now unique
non-documented command DEBUG SPLIT as a prerequisite for automatic sharding/rebalancing
Stack optimizations useful for processing complex queries:
New binlog format: you need to make a clean stop of Manticore before upgrading
Index format slightly changes: the new version can read you existing indexes fine, but if you decide to downgrade from 3.6.0 to an older version the newer indexes will be unreadable
Replication format change: don't replicate from an older version to 3.6.0 and vice versa, switch to the new version on all your nodes at once
reverse_scan is deprecated. Make sure you don't use this option in your queries since 3.6.0 since they will fail otherwise
No more implicit sorting by id. If you rely on it make sure to update your queries accordingly
Search option reverse_scan has been deprecated
New Python, Javascript and Java clients are generally available now and are well documented in this manual.
indexer --verbose is deprecated as it never added anything to the indexer output
For dumping watchdog's backtrace signal USR2 is now to be used instead of USR1
If no replication listen directive is declared, the engine will try to use ports after the defined 'sphinx' port, up to 200.
listen=...:sphinx needs to be explicit set for SphinxSE connections or SphinxAPI clients.
SQL command status now outputs Queue\Threads and Tasks\Threads.
dist_threads is completely deprecated now, searchd will log a warning if the directive is still used.
The official Docker image is now based on Ubuntu 20.04 LTS
Besides the usual manticore package, you can also install Manticore Search by components:
manticore-server-core - provides searchd, manpage, log dir, API and galera module. It will also install manticore-common as the dependency.
manticore-server - provides automation scripts for core (init.d, systemd), and manticore_new_cluster wrapper. It will also install manticore-server-core as the dependency.
manticore-common - provides config, stopwords, generic docs and skeleton folders (datadir, modules, etc.)
manticore-tools - provides auxiliary tools ( indexer, indextool etc.), their manpages and examples. It will also install manticore-common as the dependency.
manticore-icudata (RPM) or manticore-icudata-65l (DEB) - provides ICU data file for icu morphology usage.
manticore-devel (RPM) or manticore-dev (DEB) - provides dev headers for UDFs.
any highlighting that works with several fields (highlight({},'field1, field2') or highlight in json queries) now applies limits per-field by default.
any highlighting that works with plain text (highlight({}, string_attr) or snippet() now applies limits to the whole document.
SSL and compression support for SQL interface
Improved SHOW THREADS output.
Display progress of long CALL PQ in SHOW THREADS.
SET [GLOBAL] wait_timeout=NUM implemented ,
Index format has been changed. Indexes built in 3.5.0 cannot be loaded by Manticore version < 3.5.0, but Manticore 3.5.0 understands older formats.
Only absolute paths are allowed for external files (stopwords, exceptions etc.) in CREATE TABLE/ALTER TABLE.
ram_chunks_count was renamed to ram_chunk_segments_count in SHOW INDEX STATUS.
workers is obsolete. There's only one workers mode now.
dist_threads is obsolete. All queries are as much parallel as possible now (limited by threads and jobs_queue_size).
All /json/* endpoints are now available w/o /json/, e.g. /search, /insert, /delete, /pq etc.
field meaning "full-text field" was renamed to "text" in describe. 3.4.2:
3.5.0:
Cyrillic и doesn't map to i in non_cjk charset_table (which is a default) as it affected Russian stemmers and lemmatizers too much.
Ubuntu Focal 20.04 official package
deb package name changed from manticore-bin to manticore
highlight() sometimes hangs
max_query_time slows down SELECTs too much
server works in 2 modes: rt-mode and plain-mode
rt-mode requires data_dir and no index definition in config
in plain-mode indexes are defined in config; no data_dir allowed
replication available only in rt-mode
charset_table defaults to non_cjk alias
in rt-mode full-text fields are indexed and stored by default
full-text fields in rt-mode renamed from 'field' to 'text'
ALTER RTINDEX is renamed to ALTER TABLE
TRUNCATE RTINDEX is renamed to TRUNCATE TABLE
stored-only fields
SHOW CREATE TABLE, IMPORT TABLE
much faster lockless PQ
/sql can execute any type of SQL statement in mode=raw
alias mysql for mysql41 protocol
default state.sql in data_dir
Parallel Real-Time index searching
EXPLAIN QUERY command
configuration file without index definitions (alpha version)
CREATE/DROP TABLE commands (alpha version)
indexer --print-rt - can read from a source and print INSERTs for a Real-Time index
Updated to Snowball 2.0 stemmers
LIKE filter for SHOW INDEX STATUS
improved memory usage for high max_matches
SHOW INDEX STATUS adds ram_chunks_count for RT indexes
lockless PQ
changed LimitNOFILE to 65536
Autoincrement ID for RT indexes
Highlight support for docstore via new HIGHLIGHT() function, available also in HTTP API
SNIPPET() can use special function QUERY() which returns current MATCH query
new field_separator option for highlighting functions.
lazy fetch of stored fields for remote nodes (can significantly increase performance)
strings and expressions don't break anymore multi-query and FACET optimizations
RHEL/CentOS 8 build now uses mysql libclient from mariadb-connector-c-devel
ICU data file is now shipped with the packages, icu_data_dir removed
systemd service files include 'Restart=on-failure' policy
indextool can now check real-time indexes online
default conf is now /etc/manticoresearch/manticore.conf
service on RHEL/CentOS renamed to 'manticore' from 'searchd'
removed query_mode and exact_phrase snippet's options
Document storage
new directives stored_fields, docstore_cache_size, docstore_block_size, docstore_compression, docstore_compression_level
improved SSL support
non_cjk built-in charset updated
disabled UPDATE/DELETE statements logging a SELECT in query log
RHEL/CentOS 8 packages
Experimental SSL support for HTTP API
field filter for CALL KEYWORDS
max_matches for /json/search
automatic sizing of default Galera gcache.size
improved FreeBSD support
replication for RealTime indexes
ICU tokenizer for chinese
new morphology option icu_chinese
new directive icu_data_dir
multiple statements transactions for replication
LAST_INSERT_ID() and @session.last_insert_id
LIKE 'pattern' for SHOW VARIABLES
Multiple documents INSERT for percolate indexes
Added time parsers for config
internal task manager
mlock for doc and hit lists components
jail snippets path
RLP library support dropped in favor of ICU; all rlp* directives removed
updating document ID with UPDATE is disabled
added mmap readers for docs and hit lists
/sql HTTP endpoint response is now the same as /json/search response
new directives access_plain_attrs, access_blob_attrs, access_doclists, access_hitlists
new directive server_id for replication setups
removed HTTP /search endpoint
read_buffer, ondisk_attrs, ondisk_attrs_default, mlock are replaced by access_* directives
New index storage. Non-scalar attributes are not limited anymore to 4GB size per index
attr_update_reserve directive
String,JSON and MVAs can be updated using UPDATE
killlists are applied at index load time
killlist_target directive
multi AND searches speedup
better average performance and RAM usage
convert tool for upgrading indexes made with 2.x
CONCAT() function
JOIN CLUSTER cluster AT 'nodeaddress:port'
ALTER CLUSTER posts UPDATE nodes
node_address directive
list of nodes printed in SHOW STATUS
in case of indexes with killists, server doesn't rotate indexes in order defined in conf, but follows the chain of killlist targets
order of indexes in a search no longer defines the order in which killlists are applied
Document IDs are now signed big integers
docinfo (always extern now), inplace_docinfo_gap, mva_updates_pool
Galera replication for percolate indexes
OPTION morphology
Cmake minimum version is now 3.13. Compiling requires boost and libssl development libraries.
SUBSTRING_INDEX()
SENTENCE and PARAGRAPH support for percolate queries
systemd generator for Debian/Ubuntu; also added LimitCORE to allow core dumping
Distributed indexes for percolate indexes
CALL PQ new options and changes:
skip_bad_json
mode (sparsed/sharded)
json documents can be passed as a json array
shift
Column names 'UID', 'Documents', 'Query', 'Tags', 'Filters' were renamed to 'id', 'documents', 'query', 'tags', 'filters'
DESCRIBE pq TABLE
SELECT FROM pq WHERE UID is not possible any more, use 'id' instead
SELECT over pq indexes is on par with regular indexes (e.g. you can filter rules via REGEX())
ANY/ALL can be used on PQ tags
expressions have auto-conversion for JSON fields, not requiring explicit casting
built-in 'non_cjk' charset_table and 'cjk' ngram_chars
built-in stopwords collections for 50 languages
multiple files in a stopwords declaration can also be separated by comma
CALL PQ can accept JSON array of documents
REGEX function
limit/offset for json API search
profiler points for qcache
SHOW THREADS in case of remote distributed indexes prints the original query instead of API call
SHOW THREADS new option format=sphinxql prints all queries in SQL format
SHOW PROFILE prints additional clone_attrs stage
sort_mode option for CALL KEYWORDS
DEBUG on VIP connection can perform 'crash ' for intentional SIGEGV action on server
DEBUG can perform 'malloc_stats' for dumping malloc stats in searchd.log 'malloc_trim' to perform a malloc_trim()
improved backtrace is gdb is present on the system
compatibility with MySQL 8 clients
retired memory counter on SHOW STATUS for RT indexes
global cache of multi agents
improved IOCP on Windows
VIP connections for HTTP protocol
new stats to SHOW AGENT STATUS (_ping, _has_perspool, _need_resolve)
--verbose option of indexer now accept [debugvv] for printing debug messages
improved wildcards performance on matching multiple documents at PQ
support for fullscan queries at PQ
support for MVA attributes at PQ
regexp and RLP support for percolate indexes
reduced number of syscalls to avoid Meltdown and Spectre patches impact
internal rewrite of local index management
remote snippets refactor
full configuration reload
all node connections are now independent
proto improvements
Windows communication switched from wsapoll to IO completion ports
TFO can be used for communication between master and nodes
added docs_id option for documents called in CALL PQ.
percolate queries filter can now contain expressions
distributed indexes can work with FEDERATED
dummy SHOW NAMES COLLATE and SET wait_timeout (for better ProxySQL compatibility)
MySQL packets return now SERVER_STATUS_AUTOCOMMIT flag, adds compatibility with ProxySQL
indexer --dumpheader can dump also RT header from .meta file
cmake build script for Ubuntu Bionic
jemalloc at compilation. If jemalloc is present on system, it can be enabled with cmake flag -DUSE_JEMALLOC=1
new full-text matching operator NOTNEAR/N
LIMIT for SELECT on percolate indexes
a retry_count can be specified in agent definition and the value represents retries per agent
Added -h and -v options (help and version) to executables
Removed support for 32-bit docids from the code. Also removed all the code that converts/loads legacy indexes with 32-bit docids.
lots of minor fixes after thorough static code analysis
other minor bugfixes
In this release we've changed internal protocol used by masters and agents to speak with each other. In case you run Manticoresearch in a distributed environment with multiple instances make sure your first upgrade agents, then the masters.
GROUP N BY work correctly with MVA attributes
blackhole agents are run on separate thread to not affect master query anymore
implemented reference count on indexes, to avoid stalls caused by rotations and high load
SHA1 hashing implemented, not exposed yet externally
fixes for compiling on FreeBSD, macOS and Alpine
OR operator in WHERE clause between attribute filters
Maintenance mode ( SET MAINTENANCE=1)
Field weights can be zero or negative
COUNT DISTINCT works with facet searches
IN can be used with JSON float arrays
multi-query optimization is not broken anymore by integer/float expressions
Manticore Search is built using cmake and the minimum gcc version required for compiling is 4.7.2.
Manticore Search runs under manticore user.
Default data folder is now /var/lib/manticore/.
Default log folder is now /var/log/manticore/.
Default pid folder is now /var/run/manticore/.
Manticore branding
Resolved an issue where table join failed when two json attributes were used in the condition.
Fixed incorrect total_relation in multiqueries with .
Corrected filtering by json.string in the right table on .
Enabled use of null for all values in any POST HTTP JSON endpoints (insert/replace/bulk). In this case a default value is used.
Optimized memory consumption by adjusting allocation of network buffer for the initial socket probe.
Fixed insertion of unsigned int into bigint attribute via JSON interface.
Fixed secondary indexes to work correctly with exclude filters and pseudo_sharding enabled.
Addressed a bug in .
Resolved daemon crash on malformed _update request.
Fixed histograms' inability to handle value filters with excludes.
Fixed secondary indexes for JSON attributes failing with exclude filters over aliased expressions.
Fixed queries against distributed tables.
Enhanced handling of exclude filters on table encoding in columnar accessor.
Fixed escaping of special characters in CREATE TABLE and ALTER TABLE.
Allowed function names to be used as column names.
Fixed daemon crash on querying table settings with unknown disk chunk.
Removed date/time related tokens (and regex) from the reserved words.
Fixed an escaping issue in .
Fixed indexer crash when declaring multiple attributes or fields with the same name.
Resolved crash of daemon on incorrect transform for nested bool queries for the "compat" search related endpoints.
Fixed expansion in phrases with modifiers.
Addressed a warning message about an empty binlog not to be counted in "ubertests".
Resolved daemon crash on using or fulltext operator.
⚠️ BREAKING Removed Thai characters from internal cjk charset. Update your charset definitions accordingly: if you have cjk,non_cjk, change it to cjk,thai,non_cjk, or cont,non_cjk, where cont is the new designation for all continuous-script languages (i.e., cjk + thai). Modify existing tables using .
⚠️ BREAKING Fixed incorrect error message when a node joins a cluster with the wrong replication protocol version.
⚠️ BREAKING Added support for multiple tables in .
Resolved an issue where the unattended-upgrades utility, which automatically installs package updates on Debian-based systems, would incorrectly mark several Manticore packages, including manticore-galera, manticore-executor, and manticore-columnar-lib, for removal. This occurred due to dpkg mistakenly considering the virtual package manticore-extra as redundant. Changes were made to ensure unattended-upgrades no longer tries to remove essential Manticore components.
Fixed a crash introduced in version 6.3.4, which could occur when dealing with expressions and distributed or multiple tables.
Fixed a daemon crash or internal error upon early exit caused by max_query_time when querying multiple indexes.
Added support for .
Improved escaping of delimiters in word forms and exceptions.
Improved external files renaming on copy for CREATE and ALTER TABLE statements.
Added string comparison operators to SELECT list expressions.
Added support for null values in Elastic-like bulk requests.
Added support for mysqldump version 9.
Improved error handling in HTTP JSON queries with JSON path to the node where the error occurs.
Fixed performance degradation in wildcard queries with many matches when disk_chunks > 1.
Fixed crash in MVA MIN or MAX SELECT list expressions for empty MVA arrays.
Fixed incorrect processing of Kibana's infinite range request.
Fixed join filter on columnar attributes from the right table when the attribute is not in the SELECT list.
Fixed duplicate 'static' specifier in Manticore 6.3.2.
Fixed LEFT JOIN returning non-matching entries when MATCH() over the right table is used.
Fixed saving of disk chunk at RT index with hitless_words.
The 'aggs_node_sort' property can now be added in any order among other properties.
Fixed error on full-text vs filter order in the JSON query.
Fixed bug related to incorrect JSON response for long UTF-8 requests.
Fixed calculation of presort/prefilter expressions that depend on joined attributes.
Changed the method of calculating data size for metrics to read from the manticore.json file instead of checking the entire size of the data directory.
Added handling of validation requests from Vector.dev.
⚠️ Updated aggs.range values to be numeric.
Fixed grouping by stored check vs rset merge.
Fixed a crash in the daemon when querying with wildcard characters in an RT index using a CRC dictionary and local_df enabled.
Fixed a crash in JOIN on count(*) without GROUP BY.
Fixed JOIN not returning a warning when attempting grouping by a full-text field.
Addressed issue where adding an attribute via ALTER TABLE did not take KNN options into account.
Fixed failure in removing manticore-tools Redhat package in version 6.3.0.
Corrected issues with JOIN and multiple FACET statements returning incorrect results.
Fixed ALTER TABLE producing an error if the table is in a cluster.
Fixed the original query being passed into buddy from the SphinxQL interface.
Improved wildcard expansion in the CALL KEYWORDS for RT index with disk chunks.
Fixed hanging of incorrect /cli requests.
Resolved issues where concurrent requests to Manticore could get stuck.
Fixed hanging of drop table if exists t; create table t via /cli.
Added support for cluster:name format in the /_bulk HTTP endpoint.
Implemented data type; implemented .
(beta stage).
Implemented autodetection of date formats for fields.
Changed Manticore Search license from GPLv2-or-later to GPLv3-or-later.
Running Manticore in Windows now requires Docker to run Buddy.
Added a full-text operator.
Ubuntu Noble 24.04 support.
Revamp of time operations for better performance and new date/time functions:
- Returns current date in local timezone
- Returns the integer quarter of the year from a timestamp argument
- Returns the weekday name for a given timestamp argument
- Returns the name of the month for a given timestamp argument
- Returns the integer weekday index for a given timestamp argument
- Returns the integer day of the year for a given timestamp argument
- Returns the integer year and the day code of the first day of current week for a given timestamp argument
- Returns the number of days between two given timestamps
- Formats the date part from a timestamp argument
- Formats the time part from a timestamp argument
- Timezone used by date/time-related functions.
Added , , , and aggregates to the HTTP interface and similar expressions into SQL.
Support of Filebeat versions 8.10 - 8.11.
.
Added the ability to copy tables using the SQL statement.
Optimized the : Previously, both manual and automatic processes would first merge chunks to ensure the count did not exceed the limit, and then expunge deleted documents from all other chunks containing deleted documents. This approach was sometimes too resource-intensive and has been disabled. Now, chunk merging occurs solely according to the setting. However, chunks with a high number of deleted documents are more likely to be merged.
Added protection against loading a secondary index of a newer version.
Partial replace via .
Updated default merge buffer sizes: .spa (scalar attrs): 256KB -> 8MB; .spb (blob attrs): 256KB -> 8MB; .spc (columnar attrs): 1MB, no change; .spds (docstore): 256KB -> 8MB; .spidx (secondary indexes): 256KB buffer -> 128MB memory limit; .spi (dictionary): 256KB -> 16MB; .spd (doclists): 8MB, no change; .spp (hitlists): 8MB, no change; .spe (skiplists): 256KB -> 8MB.
Added via JSON.
Disabled PCRE.JIT due to issues with some regex patterns and no significant time benefit.
Added support for vanilla Galera v.3 (api v25) (libgalera_smm.so from MySQL 5.x).
Changed metric suffix from _rate to _rps.
Improved docs about balancer HA support.
Changed index to table in error messages; fixed bison parser error message fixup.
Support manticore.tbl as table name.
Support for running indexer via systemd (). ❤️ Thank you, for the PR.
Secondary indexes support in GEODIST().
Simplified .
Added support for the default values (agent_connect_timeout and agent_query_timeout) for create distributed table statement.
Added search query option that overrides searchd.expansion_limit.
Implemented for int->bigint conversion.
Meta information in MySQL response.
.
Support of via JSON.
Improve error "unsupported value type".
Added Buddy version into SHOW STATUS.
Match requests optimisation in case of zero docs for a keyword.
Added conversion to bool attribute from the string value of "true" and "false" on posting data.
Added table and searchd option.
Added new options: and to the searchd section of the config; made the threshold for merging tiny terms of the expanded terms configurable.
Added display of the last command time in @@system.sessions.
Upgraded Buddy Protocol to version 2.
Implemented additional request formats to the /sql endpoint to ease integration with libraries.
Added an Info section to SHOW BUDDY PLUGINS.
Improved memory consumption in CALL PQ with large packets.
Switched compiler from Clang 15 to Clang 16.
Added string comparison. ❤️ Thank you, for the PR.
Added support for joined stored fields.
Log client's host:port in query-log.
Fixed wrong error.
Introduced support for verbosity levels for the .
Disabled logging of queries from Buddy unless log_level=debug is set.
Linux Mint 21.3 suppport.
Manticore couldn't be built with Mysql 8.3+.
DEBUG DEDUP command for real-time table chunks that can experience duplicate entries after attaching a plain table containing duplicates.
Added time to .
Handle @timestamp column as timestamp.
Implemented logic to enable/disable buddy plugins.
Updated composer to a fresher version where recent CVEs are fixed.
Minor optimization in Manticore systemd unit related with RuntimeDirectory.
Added rdkafka support and updated to PHP 8.3.3.
Support of an RT table. New command .
⚠️ Fixed an IDF calculation issue. local_df is now a default. Improved the master-agent search protocol (version updated). If you are running Manticore Search in a distributed environment with multiple instances, make sure to first upgrade the agents, then the masters.
⚠️ Added replication of distributed tables and updated the replication protocol. If you are running a replication cluster, you need to:
Read about for more details.
⚠️ HTTP API endpoint aliases /json/* have been deprecated.
⚠️ Changed to in JSON, added query profiling for JSON.
⚠️ manticore-backup doesn't backup plugin_dir anymore.
⚠️ Migrated Buddy to Swoole to improve performance and stability. When switching to the new version, ensure all Manticore packages are updated.
⚠️ Merged all core plugins into Buddy and changed the core logic.
⚠️ Treating document IDs as numbers in /search responses.
⚠️ Added Swoole, disabled ZTS, and removed the parallel extension.
Fixed replication error on SST of large files.
Added a retry mechanism to replication commands; fixed replication join failure on a busy network with packet loss.
Changed the FATAL message in replication to a WARNING message.
Fixed the calculation of the gcache.page_size for replication clusters without tables or with empty tables; also fixed saving and loading of the Galera options.
Added functionality to skip the update nodes replication command on the node that joins the cluster.
Fixed deadlock during replication on updating blob attributes versus replacing documents.
Added , , , searchd config options to control network during replication similar to searchd.agent_* but with different defaults.
Fixed replication nodes retry after some nodes are missed and name resolution of these nodes failed.
Fixed the replication log verbosity level at the show variables.
Fixed a replication issue for a joiner node connecting to a cluster on a pod restarted in Kubernetes.
Fixed a long wait for replication to alter on empty cluster with an invalid node name.
Fixed unused matches cleanup in count distinct which could cause a crash.
Binary log is now written with transaction granularity.
Fixed a bug associated with 64-bit IDs that could result in a "Malformed packet" error when inserting via MySQL, leading to .
Fixed dates being inserted as if they were in UTC instead of local time zone.
Fixed a crash that occurred when performing a search in a real-time table with a non-empty index_token_filter.
Changed duplicate filtering in RT columnar storage to fix crashes and wrong query results.
Fixed html stipper corrupting memory after processing a joined field.
Avoided rewinding stream after flush to prevent miscommunication issues with mysqldump.
Don't wait for preread to finish if it has not started.
Fixed large Buddy output string to split among multiple lines in the searchd log.
Moved MySQL interface warning about failing header debugv verbosity level.
Fixed race condition on multiple clusters management operations; prohibited creating multiple clusters with the same name or path.
Fixed implicit cutoff in fulltext queries; split MatchExtended into template partD.
Fixed the discrepancy of index_exact_words between indexing and loading the table to the daemon.
Fixed missed error message for invalid cluster deletion.
Fixed CBO vs queue union; fixed CBO vs RT pseudo sharding.
When starting without the secondary index (SI) library and parameters in the configuration, the misleading warning message 'WARNING: secondary_indexes set but failed to initialize secondary library' was issued.
fixed hit sorting in quorum.
Fixed issue with upper case options in the ModifyTable plugin.
Fixed restoring from a dump with empty json values (represented as NULL).
Fixed SST timeout at the joiner node when receiving SST by using pcon.
Fixed a crash on selecting aliased string attribute.
Added query transform of the term into =term of full-text query with the morphology_skip_fields field.
Added missing config key (skiplist_cache_size).
Fixed crash at the expression ranker with large complex query.
Fixed fulltext CBO vs invalid index hints.
Interrupt preread on shutdown for faster shutdown.
Changed stack calculation for fulltext queries to avoid a crash in case of a complex query.
Fixed a crash of the indexer when indexing an SQL source with multiple columns having the same name.
Return 0 instead of for non-existing sysvars.
Fixed indextool error when checking external files of the RT table.
Fixed query parse error due to multi wordform inside the phrase.
Added the replay of empty binlog files with old binlog versions.
Fixed removal of the last empty binlog file.
Fixed incorrect relative paths (converted to absolute from the daemon's start directory) after changes in data_dir affect the current work directory on daemon start.
Slowest time degradation in hn_small: fetch/cache cpu info on daemon startup.
Fixed warning regarding missing external file during index load.
Fixed crash at global groupers on free of data ptr attributes.
_ADDITIONAL_SEARCHD_PARAMS is not working.
Fixed per table agent_query_timeout being replaced by the default query option agent_query_timeout.
Fixed crash at the grouper and ranker when using packedfactors() with multiple values per match.
Manticore crashes on frequent index updates.
Fixed crash on cleanup of the parsed query after parse error.
Fixed HTTP JSON requests not being routed to buddy.
JSON attribute root value couldn't be an array. Fixed.
Fixed crash on table recreation within the transaction.
Fixed expansion of the short forms of the RU lemmas.
Fixed JSON and STRING attributes usage in expression.
Fixed the grouper for multiple aliases to JSON fields.
Wrong total_related in dev: fixed implict cutoff vs limit; added better fullscan detection in json queries.
Fixed JSON and STRING attributes usage in all date expression.
crash on using LEVENSHTEIN().
Fixed memory corruption after a search query parse error with highlight.
Disabled wildcard expansion for terms shorter than min_prefix_len / min_infix_len.
Altered behavior to not log an error if Buddy handles the request successfully.
Fixed total at the meta of the search query for queries with limit set.
Impossible to use a table with an upper case via JSON in plain mode.
Provided a default SPH_EXTNODE_STACK_SIZE value.
Fixed SphinxQL log of negative filter with ALL/ANY on MVA attribute.
Fix application of docid killlists from other indexes. ❤️ Thank you, for the PR.
Fixed missed matches due to early exit on raw index full scan (without any indexes iterators); removed cutoff from the plain row iterator.
Fixed FACET error when querying a distributed table with agent and local tables.
Fixed crash on histogram estimation for large values.
crash on alter table tbl add column col uint.
Empty result for condition WHERE json.array IN (<value>).
Fixed an issue with TableFormatter when sending request to /cli.
CREATE TABLE wasn't failing in case of a missing wordforms file.
The order of attributes in RT tables now follows the configuration order.
HTTP bool query with 'should' condition returns incorrect results.
Sorting by string attributes does not work with SPH_SORT_ATTR_DESC and SPH_SORT_ATTR_ASC.
Disabled the Expect: 100-continue HTTP header for curl requests to Buddy.
crash caused by GROUP BY alias.
SQL meta summary shows wrong time on windows.
Fixed a single-term performance drop with JSON queries.
Incompatible filters didn't raise an error on /search.
Fixed ALTER CLUSTER ADD and JOIN CLUSTER operations to wait for each other, preventing a race condition where ALTER adds a table to the cluster while the donor sends tables to the joiner node.
Incorrect handling of /pq/{table}/* requests.
UNFREEZE wasn't working in some cases.
Fixed an issue with MVA restoration in some cases.
Fixed indextool crash on shutdown if used with MCL.
Fixed unnecessary url decoding for /cli_json requests.
change plugin_dir set logic on daemon start.
alter table ... exceptions fails.
Manticore crashes with signal 11 when inserting data.
Reduced throttling for .
Mysqldump + mysql restore bug.
Overriding in charset_table was not working in some cases.
Fixed incorrect creation of the distributed table in the case of a missing local table or incorrect agent description; now returns an error message.
Implemented a FREEZE counter to avoid freeze/unfreeze issues.
Obey query timeout in OR nodes. Previously max_query_time could be not working in some cases.
Failed to rename new to current [manticore.json].
A full-text query could ignore a SecondaryIndex CBO hint.
Fixed expansion_limit to slice final result set for call keywords from multiple disk chunks or RAM chunks.
wrong external files.
A few manticore-executor processes could be left running after stopping Manticore.
Crash using Levenshtein Distance.
Got error after multiple max operator ran on an empty index.
crash on multi-group with JSON.field.
Manticore was crashing on incorrect request to _update.
Fixed an issue with string filter comparators for closed ranges in the JSON interface.
alter failed when the data_dir path was located on a symlink.
Improved special handling of SELECT queries in mysqldump to ensure the resulting INSERT statements are compatible with Manticore.
Thai chars were in the wrong charsets.
Crash if I use an SQL with a reserved word.
Tables with wordforms couldn't be imported.
Fixed a crash that occurred when the engine parameter was set to 'columnar' and duplicate IDs were added via JSON.
Proper error when trying to insert a document w/o schema and w/o column names.
Auto-schema multi-line insert could fail.
Added an error message on indexing if an id attribute is declared at the data source.
Manticore cluster breakdown.
optimize.php would crash if percolate table was present.
Fixed errors when deploying on Kubernetes.
Fixed the incorrect processing of concurrent requests to Buddy
Set VIP HTTP port as default when available. Various improvements: improved versions check and streaming ZSTD decompression; added user prompts for version mismatches during restore; fixed incorrect prompting behavior for different versions on restore; enhanced decompression logic to read directly from the stream rather than into working memory; added --force flag
Added backup version display after Manticore search start to identify issues at this stage.
Updated error message for failed connections to the daemon.
Fixed issue with converting absolute root backup paths to relative and removed writeable check on restore to enable restoration from different paths.
Added sorting to the file iterator to ensure consistency across various situations.
Backup and restore of multiple configurations.
Added defattr to prevent unusual user permissions in files after installation on RHEL.
Added extra chown to ensure files default to the root user in Ubuntu.
Vector search support.
Fixed cleanup of temporary files during the interrupted setup of the secondary index build. This resolves the issue where the daemon exceeded the open files limit when creating tmp.spidx files.
Use separate streamvbyte library for columnar and SI.
Added a warning that columnar storage doesn't support json attrs.
Fixed data unpacking in SI.
Fixed a crash on saving a disk chunk with mixed rowwise and columnar storage.
Fixed SI iterator being hinted at an already processed block.
Update is broken for rowwise MVA column with columnar engine.
Fixed crash when aggregating to a columnar attribute used in HAVING.
Fixed crash in expr ranker on using columnar attribute.
❗ Support of via environment variables.
❗ Improved flexibility of configuration via environment vars.
Improved the processes for Docker.
Improved entrypoint to handle backup restoration on first start only.
Fixed query logging to stdout.
Mute BUDDY warnings if EXTRA is not set.
Fixed hostname in manticore.conf.sh.
❗ "Manticore 6.2.0 doesn't start via systemctl on Centos 7": Modified TimeoutStartSec from infinity to 0 for better compatibility with Centos 7.
❗ "Crash after upgrading from 6.0.4 to 6.2.0": Added replay functionality for empty binlog files from older binlog versions.
"fix typo in searchdreplication.cpp": Corrected a typo in searchdreplication.cpp: beggining -> beginning.
"Manticore 6.2.0 WARNING: conn (local)(12), sock=8088: bailing on failed MySQL header, AsyncNetInputBuffer_c::AppendData: error 11 (Resource temporarily unavailable) return -1": Lowered the verbosity level of the MySQL interface warning about the header to logdebugv.
"join cluster hangs when node_address can't be resolved": Improved replication retry when certain nodes are unreachable, and their name resolution fails. This should resolve issues in Kubernetes and Docker nodes related to replication. Enhanced the error message for replication start failures and made updates to test model 376. Additionally, provided a clear error message for name resolution failures.
"No lower case mapping for "Ø" in charset non_cjk": Adjusted the mapping for the 'Ø' character.
"searchd leaves binlog.meta and binlog.001 after clean stop": Ensured that the last empty binlog file is removed properly.
: Fixed the Thd_t build issue on Windows related to atomic copy restrictions.
: Addressed an issue with FT CBO vs ColumnarScan.
: Made corrections to test 376 and added a substitution for the AF_INET error in the test.
: Resolved a deadlock issue during replication when updating blob attributes versus replacing documents. Also removed the rlock of the index during commit because it's already locked at a more basic level.
Updated info on /bulk endpoints in the manual.
Support of v2.2.4
- to using mysqldump
and to visualize data stored in Manticore
and for easier development with Manticore
We've started using , making it simpler for contributors to utilize the same Continuous Integration (CI) process that the core team applies when preparing packages. All jobs can be run on GitHub-hosted runners, which facilitates seamless testing of changes in your fork of Manticore Search.
We've started using to test complex scenarios. For example, we're now able to ensure that a package built after a commit can be properly installed across all supported Linux operating systems. The Command Line Tester (CLT) provides a user-friendly way to record tests in an interactive mode and to effortlessly replay them.
Users now have the option to specify via the configuration settings, providing better customization to match specific workload requirements.
Support for with numerous bug fixes and improvements in .
: The HTTP endpoint now serves as an alias for the /json/pq HTTP endpoint.
: We've ensured multi-byte compatibility for upper() and lower().
: Instead of scanning the index for count(*) queries, a precalculated value is now returned.
: It's now possible to use SELECT for making arbitrary calculations and displaying @@sysvars. Unlike before, you are no longer limited to just one calculation. Therefore, queries like select user(), database(), @@version_comment, version(), 1+1 as a limit 10 will return all the columns. Note that the optional 'limit' will always be ignored.
: Implemented the CREATE DATABASE stub query.
: When executing ALTER TABLE table REBUILD SECONDARY, secondary indexes are now always rebuilt, even if attributes weren't updated.
: Sorters utilizing precalculated data are now identified before using CBO to avoid unnecessary CBO calculations.
: Implementing mocked and utilizing of the full-text expression stack to prevent daemon crashes.
: A speedy code path has been added for match cloning code for matches that don't use string/mvas/json attributes.
: Added support for the SELECT DATABASE() command. However, it will always return Manticore. This addition is crucial for integrations with various MySQL tools.
: Modified the response format of the endpoint, and added the /cli_json endpoint to function as the previous /cli.
: The thread_stack can now be altered during runtime using the SET statement. Both session-local and daemon-wide variants are available. Current values can be accessed in the show variables output.
: Code has been integrated into CBO to more accurately estimate the complexity of executing filters over string attributes.
: The DocidIndex cost calculation has been improved, enhancing overall performance.
: Load metrics, similar to 'uptime' on Linux, are now visible in the SHOW STATUS command.
: The field and attribute order for DESC and SHOW CREATE TABLE now match that of SELECT * FROM.
: Different internal parsers now provide their internal mnemonic code (e.g., P01) during various errors. This enhancement aids in identifying which parser caused an error and also obscures non-essential internal details.
"Sometimes CALL SUGGEST does not suggest a correction of a single letter typo": Improved behaviour for short words: added the option sentence to show the entire sentence
"Percolate index does not search properly by exact phrase query when stemming enabled": The percolate query has been modified to handle an exact term modifier, improving search functionality.
"DATE FORMATTING methods": added the select list expression, which exposes the strftime() function.
"Sorting buckets via HTTP JSON API": introduced an optional for each bucket of aggregates in the HTTP interface.
"Improve error logging of JSON insert api failure - "unsupported value type"": The /bulk endpoint reports information regarding the number of processed and non-processed strings (documents) in case of an error.
"CBO hints don't support multiple attributes": Enabled index hints to handle multiple attributes.
"Add tags to http search query": Tags have been added to .
"buddy should not create table in parallel": Resolved an issue that was causing failures from parallel CREATE TABLE operations. Now, only one CREATE TABLE operation can run at a time.
"add support of @ to column names".
"Queries on taxi dataset are slow with ps=1": The CBO logic has been refined, and the default histogram resolution has been set to 8k for better accuracy on attributes with randomly distributed values.
"Fix CBO vs fulltext on hn dataset": Enhanced logic has been implemented for determining when to use bitmap iterator intersection and when to use a priority queue.
"columnar: change iterator interface to single-call" : Columnar iterators now use a single Get call, replacing the previous two-step AdvanceTo + Get calls to retrieve a value.
"Aggregate calc speedup (remove CheckReplaceEntry?)": The CheckReplaceEntry call was removed from the group sorter to expedite the calculation of aggregate functions.
"create table read_buffer_docs/hits doesn't understand k/m/g syntax": The CREATE TABLE options read_buffer_docs and read_buffer_hits now support k/m/g syntax.
for English, German and Russian can now be effortlessly installed on Linux by executing the command apt/yum install manticore-language-packs. On macOS, use the command brew install manticoresoftware/tap/manticore-language-packs.
The type conversion function has been added.
The /bulk endpoint now processes empty lines as a command. More info .
Warnings have been implemented for , providing more transparency and allowing for error mitigation.
⚠️ : The usage of @ in table names has been disallowed to prevent syntax conflicts.
⚠️ : MCL libraries will no longer load on systems that don't support SSE 4.2.
⚠️ : was broken. Fixed and is now effective.
"Crash on DROP TABLE": resolved a problem causing extended wait times to finish write operations (optimize, disk chunk save) on an RT table when executing a DROP TABLE statement. Added a warning to notify when a table directory is not empty after executing a DROP TABLE command.
: Support for columnar attributes, which was missing in the code used for grouping by multiple attributes, has been added.
Resolved a crash issue potentially caused by disk space running out by properly handling write errors in binlog.
: A crash that occasionally occurred when using multiple columnar scan iterators (or secondary index iterators) in a query has been fixed.
: Filters were not being removed when using sorters that use precalculated data. This issue has been fixed.
: The CBO code has been updated to provide better estimates for queries using filters over row-wise attributes executed in multiple threads.
, "fatal crash dump in Kubernetes cluster": Fixed a defective bloom filter for the JSON root object; fixed daemon crash due to filtering by a JSON field.
Rectified daemon crash caused by invalid manticore.json config.
Fixed the json range filter to support int64 values.
.sph files could be corrupted ALTER. Fixed.
: A shared key has been added for the replication of the replace statement to resolve a pre_commit error occurring when replace is replicated from multiple master nodes.
resolved issues with bigint checks over functions like 'date_format()'.
: Iterators are no longer displayed in when sorters utilize precalculated data.
: The fulltext node stack size has been updated to prevent crashes on complex fulltext queries.
: A bug causing a crash during the replication of updates with JSON and string attributes has been resolved.
: The string builder has been updated to use 64-bit integers to avoid crashes when dealing with large data sets.
: Addressed a crash that was occurring with count distinct across multiple indexes.
: Fixed an issue where queries over disk chunks of RT indexes could be executed in multiple threads even if pseudo_sharding was disabled.
The set of values returned by the show index status command has been modified and now varies depending on the type of index in use.
Fixed an HTTP error when processing bulk requests and an issue where the error wasn't being returned to the client from the net loop.
use of an extended stack for PQ.
Updated the export ranker output to align with .
: Fixed an issue with the string list in the filter of the SphinxQL query log.
"The charset definition seems to depend on the ordering of codes": Fixed incorrect charset mapping for duplicates.
"Mapping multiple words in word forms interferes phrase search with CJK punctuations between keywords": Fixed ngram token position within phrase query with wordforms.
"Equals sign in search query breaks request": Ensured the exact symbol can be escaped and fixed double exact expansion by the expand_keywords option.
"exceptions/stopwords conflict"
"Manticore crash when calling call snippets() with libstemmer_fr and index_exact_words": Resolved internal conflicts causing crashes when SNIPPETS() was called.
"Duplicate records during SELECT": Fixed the issue of duplicate documents in the result set for a query with not_terms_only_allowed option to RT index with killed documents.
"Using JSON arguments in UDF functions leads to a crash": Fixed a daemon crash when processing a search with pseudo-sharding enabled and UDF with JSON argument.
"count(*) in FEDERATED": Fixed a daemon crash occurring with a query through a FEDERATED engine with aggregate.
Fixed an issue where rt_attr_json column was incompatible with columnar storage.
"* is removed from search query by ignore_chars": Fixed this issue so wildcards in a query aren't impacted by ignore_chars.
"indextool --check fails if there's a distributed table": indextool is now compatible with instances having 'distributed' and 'template' indexes in the json config.
"particular select on particular RT dataset leads to crash of searchd": Resolved daemon crash on a query with packedfactors and large internal buffer.
"With not_terms_only_allowed deleted documents are ignored"
"indextool --dumpdocids is not working": Restored functionality of the --dumpdocids command.
"indextool --buildidf is not working": indextool now closes the file after finishing globalidf.
"Count(*) is trying to be treated as schema set in remote tables": Resolved an issue where an error message was being sent by the daemon for queries into the distributed index when the agent returned an empty result set.
"FLUSH ATTRIBUTES hangs with threads=1".
"Lost connection to MySQL server during query - manticore 6.0.5": Crashes that were happening when using multiple filters over columnar attributes have been addressed.
"JSON string filtering case sensitivity": Corrected the collation to function correctly for filters used in HTTP search requests.
"Match in a wrong field": Fixed the damage related with morphology_skip_fields.
"system remote commands via API should pass g_iMaxPacketSize": Made updates to bypass the max_packet_size check for replication commands between nodes. Additionally, the latest cluster error has been added to the status display.
"tmp files left on failed optimize": Corrected an issue where temporary files were left behind after an error occurred during a merge or optimize process.
"add env var for buddy start timeout": Added environment variable MANTICORE_BUDDY_TIMEOUT (default 3 seconds) to control the daemon's wait duration for a buddy message at startup.
"Int overflow when saving PQ meta": Mitigated excessive memory consumption by daemon on saving large PQ index.
"Can't recreate RT table after altering its external file": Rectified an error of alter with empty string for external files; fixed RT index external files left after altering external files.
"SELECT statement sum(value) as value doesn't work properly": Fixed issue where select list expression with alias could hide index attribute; also fixed sum to count in int64 for integer.
"Avoid binding to localhost in replication": Ensured replication doesn't bind to localhost for host names with multiple IPs.
"reply to mysql client failed for data larger 16Mb": Fixed the issue of returning a SphinxQL packet larger than 16Mb to the client.
"wrong reference in "paths to external files should be absolute": Corrected the display of the full path to external files in SHOW CREATE TABLE.
"debug build crashes on long strings in snippets": Now, long strings (>255 characters) are permitted in the text targeted by the SNIPPET() function.
"spurious crash on use-after-delete in kqueue polling (master-agent)": Fixed crashes when the master cannot connect to the agent on kqueue-driven systems (FreeBSD, MacOS, etc.).
"too long connect to itself": When connecting from the master to agents on MacOS/BSD, a unified connect+query timeout is now used instead of just connect.
"pq (json meta) with unreached embedded synonyms fails to load": Fixed the embedded synonyms flag in pq.
"Allow some functions (sint, fibonacci, second, minute, hour, day, month, year, yearmonth, yearmonthday) to use implicitly promoted argument values".
"Enable multithreaded SI in fullscan, but limit threads": Code has been implemented into CBO to better predict multithreaded performance of secondary indexes when they're utilized in a full-text query.
"count(*) queries still slow after using precalc sorters": Iterators are no longer initiated when employing sorters that use precalculated data, circumventing detrimental performance effects.
"query log in sphinxql does not preserve original queries for MVA's": Now, all()/any() is logged.
Log Buddy version on Manticore start.
, Fixed bad character at the search meta and call keywords for bigram index.
Lowercase HTTP headers are rejected.
❗ Fixed memory leak at daemon on reading output of the Buddy console.
Fixed unexpected behavior of question mark.
- Fixed race condition in tokenizer lowercase tables causing a crash.
Fixed bulk writes processing in the JSON interface for documents with id explicitly set to null.
Fixed term statistics in CALL KEYWORDS for multiple same terms.
Default config is now created by Windows installer; paths are no longer substituted in runtime.
, Fixed replication issues for cluster with nodes in multiple networks.
Fixed /pq HTTP endpoint to be an alias of the /json/pq HTTP endpoint.
Fixed daemon crash on Buddy restart.
Display original error on invalid request received.
Allow spaces in backup path and add some magic to regexp to support single quotes also.
Crash / Segmentation Fault on Facet search with larger number of results
❗ - WARNING: Compiled-in value KNOWN_CREATE_SIZE (16) is less than measured (208). Consider to fix the value!
❗ - Manticore 6.0.0 plain index crashes
❗ - multiple distributed lost on daemon restart
❗ - race condition in tokenizer lowercase tables
Starting with this release, Manticore Search comes with Manticore Buddy, a sidecar daemon written in PHP that handles high-level functionality that does not require super low latency or high throughput. Manticore Buddy operates behind the scenes, and you may not even realize it is running. Although it is invisible to the end user, it was a significant challenge to make Manticore Buddy easily installable and compatible with the main C++-based daemon. This major change will allow the team to develop a wide range of new high-level features, such as shards orchestration, access control and authentication, and various integrations like mysqldump, DBeaver, Grafana mysql connector. For now it already handles , and .
🔬 Experimental: you can now execute Elasticsearch-compatible and JSON queries which enables using Manticore with tools like Logstash (version < 7.13), Filebeat and other tools from the Beats family. Enabled by default. You can disable it using SET GLOBAL ES_COMPAT=off.
Support for with numerous fixes and improvements in . ⚠️ BREAKING CHANGE: Secondary indexes are ON by default as of this release. Make sure you do if you are upgrading from Manticore 5. See below for more details.
Auto-schema: you can now skip creating a table, just insert the first document and Manticore will create the table automatically based on its fields. Read more about this in detail . You can turn it on/off using .
Vast revamp of which lowers query response time in many cases.
Parallelization performance estimate in CBO.
CBO is now aware of and can act smarter.
Encoding stats of columnar tables/fields are now stored in the meta data to help CBO make smarter decisions.
Added CBO hints for fine-tuning its behaviour.
: we are excited to announce the addition of telemetry in this release. This feature allows us to collect anonymous and depersonalized metrics that will help us improve the performance and user experience of our product. Rest assured, all data collected is completely anonymous and will not be linked to any personal information. This feature can be in the settings if desired.
to rebuild secondary indexes whenever you want, for example:
when you did (i.e. ) of an attribute in the index
New tool manticore-backup for
SQL command to do backups from inside Manticore.
SQL command as an easy way to see running queries rather than threads.
SQL command KILL to kill a long-running SELECT.
SQL commands to prepare a real-time/plain table for a backup.
New settings accurate_aggregation and max_matches_increase_threshold for controlled aggregation accuracy.
Support for signed negative 64-bit IDs. Note, you still can't use IDs > 2^63, but you can now use ids in the range of from -2^63 to 0.
Refactoring of all related to time scheduling as a prerequisite for parallel chunks merging.
If you are running a replication cluster, you'll need to run ALTER TABLE <table name> REBUILD SECONDARY on all the nodes or follow with just change: run the ALTER .. REBUILD SECONDARY instead of the OPTIMIZE.
SHOW SETTINGS: you can now see the settings from the configuration file from inside Manticore.
turns on/off cpu time tracking; now doesn't show CPU statistics when the cpu time tracking is off.
RT table RAM chunk segments can now be merged while the RAM chunk is being flushed.
Added secondary index progress to the output of .
Previously a table record could be removed by Manticore from the index list if it couldn't start serving it on start. The new behaviour is to keep it in the list to try to load it on the next start.
returns all the words and hits belonging to requested document.
Environment variable dump_corrupt_meta enables dumping a corrupted table meta data to log in case searchd can't load the index.
DEBUG META can show max_matches and pseudo sharding statistics.
A better error instead of the confusing "Index header format is not json, will try it as binary...".
Ukirainian lemmatizer path has been changed.
Secondary indexes statistics has been added to .
JSON interface can now be easily visualized using Swagger Editor https://manual.manticoresearch.com/Openapi#OpenAPI-specification.
read about for more details.
Manticore Columnar Library optimization which can lower response time by partial preliminary min/max evaluation.
If a disk chunk merge is interrupted, the daemon now cleans up the MCL-related tmp files.
Columnar and secondary libraries versions are dumped to log on crash.
Added support for quick doclist rewinding to secondary indexes.
Queries like select attr, count(*) from plain_index (w/o filtering) are now faster in case you are using MCL.
@@autocommit in HandleMysqlSelectSysvar for compatibility with .net connector for mysql greater than 8.25
⚠️ BREAKING CHANGE: MCL: add SSE code to columnar scan. MCL now requires at least SSE4.2.
⚠️ BREAKING CHANGE: Support for Debian Stretch and Ubuntu Xenial has been discontinued.
Debian Bookworm support.
Packaging: arm64 builds for Linuxes and MacOS.
Multi-architecture (x86_64 / arm64) docker image.
.
Windows installer (previously we provided just an archive).
Field with name text
id can't be non bigint
ALTER vs field with name "text"
❗ Possible BUG: HTTP (JSON) offset and limit affects facet results
❗ Searchd hangs/crashes under intensive loading
❗ PQ index out of memory
❗ binlog_flush = 1 has been broken all the time since Sphinx. Fixed.
MCL: crash on select when too many ft fields
sql_joined_field can't be stored
Crash when using LEVENSHTEIN()
Manticore crashes unexpected and cant to normal restart
CALL KEYWORDS through /sql returns control char which breaks json
mariadb can't create table FEDERATED
WARNING: dlopen() failed: /usr/bin/lib_manticore_columnar.so: cannot open shared object file: No such file or directory
Manticore crashes when search with ZONESPAN is done through api
wrong weight when using multiple indexes and facet distinct
SphinxQL group query hangs after SQL index reprocessing
MCL: Indexer crashes in 5.0.2 and manticore-columnar-lib 1.15.4
Manticore 5.0.2 FEDERATED returns empty set (MySQL 8.0.28)
select COUNT DISTINCT on 2 indices when result is zero throws internal error
CRASH on delete query
MCL: Bug with long text field
5.0.2 rtindex: Aggregate search limit behavior is not as expected
Hits returned is Nonetype object even for searches that should return multiple results
Crash with using Attribute and Stored Field in SELECT expression
table gets invisible after crash
Two negative terms in search query gives error: query is non-computable
a -b -c is not working via json query_string
pseudo_sharding with query match
Manticore 5.0.2 min/max function doesn't work as expecting ...
Field "weight" is not parsed correctly
Manticore service crash upon start and keep restarting
group by j.a, smth works wrong
Searchd crash when expr used in ranker, but only for queries with two proximities
net_throttle_action is broken
MCL: Manticore crashes on query execution and other crashed during cluster recovery.
SHOW CREATE TABLE outputs w/o backticks
It's now possible to query Manticore from Java via JDBC connector
bm25f ranking problems
configless indexes frozen in watchdog on the first-load state
Segfault when sorting facet data
crash on CONCAT(TO_STRING)
In some cases a single simple select could cause the whole instance stall, so you couldn't log in to it or run any other query until the running select is done.
Indexer crash
wrong count from facet distinct
LCS is calculating incorrectly in built-in sph04 ranker
5.0.3 dev crashing
FACET with json on engine columnar crash
MCL: 5.0.3 crash from secondary index
@@autocommit in HandleMysqlSelectSysvar
Fix thread-chunk distribution in RT indexes
Fix thread-chunk distribution in RT indexes
wrong default max_query_time
Crash on when using regex expression in multithreaded execution
Broken backward index compatibility
indextool reports error checking columnar attributes
memleak of json grouper clones
Memleak of levenshtein func cloning
Error message lost when loading meta
Propagate errors from dynamic indexes/subkeys and sysvars
Crash on count distinct over a columnar string in columnar storage
MCL: min(pickup_datetime) from taxi1 crashes
empty excludes JSON query removes columns from select list
Secondary tasks run on current scheduler sometimes cause abnormal side effects
crash with facet distinct and different schemas
MCL: Columnar rt index became damaged after run without columnar library
implicit cutoff is not working in json
Columnar grouper issue
Unable to delete last field from the index
wrong behaviour after --new-cluster
"columnar library not loaded", but it's not required
no error for delete query
Fixed ICU data file location in Windows builds
Handshake send problem
Display id in show create table
Crash / Segmentation Fault on Facet search with larger number of results.
RT index: searchd "stuck" forever when many documents are being inserted and RAMchunk gets full
Thread gets stuck on shutdown while replication is busy between nodes
Mixing floats and ints in a JSON range filter could make Manticore ignore the filter
Float filters in JSON were inaccurate
Discard uncommitted txns if index altered (or it can crash)
Query syntax error when using backslash
workers_clients could be wrong in SHOW STATUS
fixed a crash on merging ram segments w/o docstores
Fixed missed ALL/ANY condition for equals JSON filter
Replication could fail with got exception while reading ist stream: mkstemp(./gmb_pF6TJi) failed: 13 (Permission denied) if the searchd was started from a directory it can't write to.
Since 4.0.2 crash log included only offsets. This commit fixes that.
❗ - wrong stack size could cause a crash.
🔬 Support for , which enables beta version. Building secondary indexes is on by default for plain and real-time columnar and row-wise indexes (if is in use), but to enable it for searching you need to set secondary_indexes = 1 either in your configuration file or using . The new functionality is supported in all operating systems except old Debian Stretch and Ubuntu Xenial.
: you can now specify listeners that process only read queries discarding any writes.
New endpoint for running SQL queries over HTTP even easier.
support in JSON protocol. Previously you couldn't code things like a=1 and (b=2 or c=3) in JSON: must (AND), should (OR) and must_not (NOT) worked only on the highest level. Now they can be nested.
Support for in the HTTP protocol. You can now use chunked transfer in your application to transmit large batches with reduced resource consumption (since calculating Content-Length is unnecessary). On the server side, Manticore now always processes incoming HTTP data in a streaming manner, without waiting for the entire batch to be transferred as before, which:
allows you to bypass and transfer batches much larger than the maximum allowed value of max_packet_size (128MB), for example, 1GB at a time.
HTTP interface support of 100 Continue: now you can transfer large batches from curl (including curl libraries used by various programming languages) which by default does Expect: 100-continue and waits some time before actually sending the batch. Previously you had to add Expect: header, now it's not needed.
⚠️ BREAKING CHANGE: is enabled by default. If you want to disable it make sure you add pseudo_sharding = 0 to section searchd of your Manticore configuration file.
for attributes backed by : queries like select * from <columnar table> are now much faster than previously, especially if there are many fields in the schema.
⚠️ BREAKING CHANGE: Implicit . Manticore now doesn't spend time and resources processing data you don't need in the result set which will be returned. The downside is that it affects total_found in and in JSON output. It is now only accurate in case you see total_relation: eq while total_relation: gte means the actual number of matching documents is greater than the total_found value you've got. To retain the previous behaviour you can use search option cutoff=0, which makes total_relation always eq.
⚠️ BREAKING CHANGE: All full-text fields are now by default. You need to use stored_fields = (empty value) to make all fields non-stored (i.e. revert to the previous behaviour).
HTTP JSON supports .
⚠️ BREAKING CHANGE: Session state support with help of . This makes HTTP stateful when the client supports it too. For example, using the new endpoint and HTTP keep-alive (which is on by default in all browsers) you can call SHOW META after SELECT and it will work the same way it works via mysql. Note, previously Connection: keep-alive HTTP header was supported too, but it only caused reusing the same connection. Since this version it also makes the session stateful.
You can now specify columnar_attrs = * to define all your attributes as columnar in the which is useful in case the list is long.
read about for more details.
DEBUG malloc_stats support for .
is now available as a per-table setting which can be set when you CREATE or ALTER a table.
⚠️ BREAKING CHANGE: is now sphinxql by default. If you are used to plain format you need to add query_log_format = plain to your configuration file.
default value was increased from 3 seconds to 60 seconds.
Support for Java mysql connector >= 6.0.3: in they changed the way they connect to mysql which broke compatibility with Manticore. The new behaviour is now supported.
disabled saving a new disk chunk on loading an index (e.g. on searchd startup).
Support for glibc >= 2.34.
count 'VIP' connections separately from usual (non-VIP). Previously VIP connections were counted towards the max_connections limit, which could cause "maxed out" error for non-VIP connections. Now VIP connections are not counted towards the limit. Current number of VIP connections can be also seen in SHOW STATUS and status.
can now be specified explicitly.
support zstd compression for mysql proto
⚠️ BM25F formula has been slightly updated to improve search relevance. This only affects search results in case you use function , it doesn't change behaviour of the default ranking formula.
⚠️ Changed behaviour of REST endpoint: /sql?mode=raw now requires escaping and returns an array.
⚠️ If you are using old , or clients please follow the corresponding link and find an updated version. The old versions are not fully compatible with Manticore 5.
Support for Amazon Linux 2 via
Random crash when using UDF function
out of memory while indexing RT index
Breaking change 3.6.0, 4.2.0 sphinxql-parser
FATAL: out of memory (unable to allocate 9007199254740992 bytes)
Strings not passed correctly to UDFs
❗ Searchd crashes after trying to add a text column to a rt index
Indexer couldn't find all columns
❗ Grouping by json.boolean works wrong
indextool commands related to index (eg. --dumpdict) failure
❗ Fields disappear from the selection
.NET HttpClient Content-Type incompatibility when using application/x-ndjson
Field length calculation
❗ create/insert into/drop columnar table has a memleak
Empty column in results under certain conditions
❗ Crash of daemon on start
❗ Daemon hangs on start
❗ Crash at SST
Json attribute marked as columnar when engine='columnar'
Replication listens on 0
columnar_attrs = * is not working with csvpipe
❗ Crash on select float in columnar in rt
❗ Indextool changes rt index during check
Need a check for listeners port range intersections
Log original error in case RT index failed to save disk chunk
Only one error reported for RE2 config
❗ RAM consumption changes in commit 5463778558586d2508697fa82e71d657ac36510f
3rd node doesn't make a non-primary cluster after dirty restart
Update counter gets increased by 2
New version 4.2.1 corrupt index created with 4.2.0 with morphology using
No escaping in json keys /sql?mode=raw
❗ Using function hides other values
❗ Memleak triggered by a line in FixupAttrForNetwork
❗ Memleak in 4.2.0 and 4.2.1 related with docstore cache
Strange ping-pong with stored fields over network
lemmatizer_base reset to empty if not mentioned in 'common' section
pseudo_sharding makes SELECT by id slower
DEBUG malloc_stats output zeros when using jemalloc
Drop/add column makes value invisible
Can't add column bit(N) to columnar table
"cluster" gets empty on start in manticore.json
❗ HTTP actions are not tracked in SHOW STATUS
disable pseudo_sharding for low frequency single keyword queries
fixed stored attributes vs index merge
generalized distinct value fetchers; added specialized distinct fetchers for columnar strings
fixed fetching null integer attributes from docstore
ranker could be specified twice in query log
Pseudo-sharding support for real-time indexes and full-text queries. In previous release we added limited pseudo sharding support. Starting from this version you can get all benefits of the pseudo sharding and your multi-core processor by just enabling . The coolest thing is that you don't need to do anything with your indexes or queries for that, just enable it and if you have free CPU it will be used to lower your response time. It supports plain and real-time indexes for full-text, filtering and analytical queries. For example, here is how enabling pseudo sharding can make most queries' response time in average about 10x lower on multiplied 100 times (116 million docs in a plain index).
is now supported.
is now available as a configuration option in section searchd. It's useful when you want to limit the RT chunks count in all your indexes to a particular number globally.
accurate and over several local physical indexes (real-time/plain) with identical fields set/order.
bigint support for YEAR() and other timestamp functions.
Adaptive . Previously Manticore Search was collecting exactly up to rt_mem_limit of data before saving a new disk chunk to disk, and while saving was still collecting up to 10% more (aka double-buffer) to minimize possible insert suspension. If that limit was also exhausted, adding new documents was blocked until the disk chunk was fully saved to disk. The new adaptive limit is built on the fact that we have now, so it's not a big deal if disk chunks do not fully respect rt_mem_limit and start flushing a disk chunk earlier. So, now we collect up to 50% of rt_mem_limit and save that as a disk chunk. Upon saving we look at the statistics (how much we've saved, how many new documents have arrived while saving) and recalculate the initial rate which will be used next time. For example, if we saved 90 million documents, and another 10 million docs arrived while saving, the rate is 90%, so we know that next time we can collect up to 90% of rt_mem_limit before starting flushing another disk chunk. The rate value is calculated automatically from 33.3% to 95%.
for PostgreSQL source. Thank you, for the .
indexer -v and --version. Previously you could still see indexer's version, but -v/--version were not supported.
infinit mlock limit by default when Manticore is started via systemd.
spinlock -> op queue for coro rwlock.
environment variable MANTICORE_TRACK_RT_ERRORS useful for debugging RT segments corruption.
new column "chain" in show threads option format=all. It shows stack of some task info tickets, most useful for profiling needs, so if you are parsing show threads output be aware of the new column.
❗ Manticore 4.0.2 slower than Manticore 3.6.3. 4.0.2 was faster than previous versions in terms of bulk inserts, but significantly slower for single document inserts. It's been fixed in 4.2.0.
❗ RT index could get corrupted under intensive REPLACE load, or it could crash
fixed average at merging groupers and group N sorter; fixed merge of aggregates
indextool --check could crash
RAM exhaustion issue caused by UPDATEs
daemon could hang on INSERT
daemon could hang on shutdown
daemon could crash on shutdown
daemon could hang on crash
daemon could crash on startup trying to rejoin cluster with invalid nodes list
distributed index could get completely forgotten in RT mode in case it couldn't resolve one of its agents on start
attr bit(N) engine='columnar' fails
create table fails, but leaves dir
Config fails with: unknown key name 'attr_update_reserve'
Manticore crash on batch queries
Batch queries causing crashes again with v4.0.3
fixed daemon crash on startup trying to re-join cluster with invalid nodes list
Manticore 4.0.2 does not accept connections after batch of inserts
FACET query with ORDER BY JSON.field or string attribute could crash
Crash SIGSEGV on query with packedfactors
morphology_skip_fields was not supported by create table
Full support of . Previously Manticore Columnar Library was supported only for plain indexes. Now it's supported:
Automatic indexes compaction (). Finally, you don't have to call OPTIMIZE manually or via a crontask or other kind of automation. Manticore now does it for you automatically and by default. You can set default compaction threshold via global variable.
Significantly faster bulk INSERT performance into a real-time index. For example on with SSD, 128 GB of RAM and AMD's Ryzen™ 9 5950X (16*2 cores) with 3.6.0 you could get 236K docs per second inserted into a table with schema name text, email string, description text, age int, active bit(1) (default rt_mem_limit, batch size 25000, 16 concurrent insert workers, 16 million docs inserted overall). In 4.0.2 the same concurrency/batch/count gives 357K docs per second.
can add/remove a full-text field (in RT mode). Previously it could only add/remove an attribute.
🔬 Experimental: pseudo-sharding for full-scan queries - allows to parallelize any non-full-text search query. Instead of preparing shards manually you can now just enable new option and expect up to CPU cores lower response time for non-full-text search queries. Note it can easily occupy all existing CPU cores, so if you care not only about latency, but throughput too - use it with caution.
Linux Mint and Ubuntu Hirsute Hippo are supported via
- added caching to lemmatizer-uk
. Now you don't need to start searchd manually in case you need to run Manticore with some specific startup flag
new function which calculates Levenshtein distance
added new --replay-flags=ignore-trx-errors and --replay-flags=ignore-all-errors so one can still start searchd if the binlog is corrupted
- expose errors from RE2
more accurate for distributed indexes consisting of local plain indexes
to remove duplicates when you do faceted search
doesn't require now and works for indexes with search enabled
ondisk_attrs_default were deprecated, now they are removed
if is not specified in a search query it gets updated implicitly with the lowest needed value for the sake of performance of the new columnar storage. It can affect metric total in , but not total_found which is the actual number of found documents.
read about for more details
- fixed crash during SST on joiner with active index; added sha1 verify at joiner node at writing file chunks to speed up index loading; added rotation of changed index files at joiner node on index load; added removal of index files at joiner node when active index gets replaced by a new index from donor node; added replication log points at donor node for sending files and chunks
- crash on JOIN CLUSTER in case the address is incorrect
- while initial replication of a large index the joining node could fail with ERROR 1064 (42000): invalid GTID, (null), the donor could become unresponsive while another node was joining
- hash could be calculated wrong for a big index which could result in replication failure
- replication failed on cluster restart
- indextool --help doesn't display parameter --rotate
- searchd high CPU usage while idle after ca. a day
- flush .meta immediately
- manticore.json gets emptied
- searchd --stopwait fails under root. It also fixes systemctl behaviour (previously it was showing failure for ExecStop and didn't wait long enough for searchd to stop properly)
- INSERT/REPLACE/DELETE vs SHOW STATUS. command_insert, command_replace and others were showing wrong metrics
- charset_table for a plain index had a wrong default value
- new disk chunks don't get mlocked
- Manticore cluster node crashes when unable to resolve a node by name
- replication of updated index can lead to undefined state
- indexer could hang on indexing a plain index source with a json attribute
- fixed not equal expression filter at PQ index
- fixed select windows at list queries above 1000 matches. SELECT * FROM pq ORDER BY id desc LIMIT 1000 , 100 OPTION max_matches=1100 was not working previously
- HTTPS request to Manticore could cause warning like "max packet size(8388608) exceeded"
- Manticore 3 could hang after a few updates of string attributes
Support for for plain indexes. New setting for plain indexes
Support for
for indexer --merge
blend_mode='trim_all'
added with backticks
can work in RT mode
for SELECT/UPDATE
, our tests show 3-4% lower latency on queries like WHERE json.a = 1
- inaccurate and unstable FACET results
- Strange behavior when using MATCH: those who suffer from this issue need to rebuild the index as the problem was on the phase of building an index
- intermittent core dump when running query with SNIPPET() function
- SELECT results in CRASH DUMP
- stack size detection for filter trees
- Update using the IN condition does not take effect correctly
- SHOW STATUS immediately after CALL PQ returns - - Fixed static binary build
- bug in multi-queries
- Unable to use unusual names for columns when use 'create table'
- daemon crash on replay binlog with update of string attribute; set binlog version to 10
- fixed expression stack frame detection runtime (test 207)
- percolate index filter and tags were empty for empty stored query (test 369)
- breaks of replication SST flow at network with long latency and high error rate (different data centers replication); updated replication command version to 1.03
- joiner lock cluster on write operations after join into cluster (test 385)
- wildcards matching with exact modifier (test 321)
- docid checkpoints vs docstore
- Inconsistent indexer behavior when parsing invalid xml
- Stored percolate query with NOTNEAR runs forever (test 349)
- wrong weight for phrase starting with wildcard
- percolate query with wildcards generate terms without payload on matching causes interleaved hits and breaks matching (test 417)
- fixed calculation of 'total' in case of parallelized query
- crash in Windows with multiple concurrent sessions at daemon
- some index settings could not be replicated
- On high rate of adding new events netloop sometimes freeze because of atomic 'kick' event being processed once for several events a time and loosing actual actions from them status of the query, not the server status
- New flushed disk chunk might be lost on commit
- inaccurate 'net_read' in profiler
- Percolate issue with arabic (right to left texts)
- id not picked correctly on duplicate column name
of network events to fix a crash in rare cases
fix in indextool --dumpheader
- TRUNCATE WITH RECONFIGURE worked wrong with stored fields
As of this release we don't provide builds for RHEL6, Debian Jessie and Ubuntu Trusty any more. If it's mission critical for you to have them supported
automatic drop of a disk chunk of a real-time index. This optimization enables dropping a disk chunk automatically when a real-time index when the chunk is obviously not needed any more (all the documents are suppressed). Previously it still required merging, now the chunk can be just dropped instantly. The option is ignored, i.e. even if nothing is actually merged an obsoleted disk chunk gets removed. This is useful in case you maintain retention in your index and delete older documents. Now compacting such indexes will be faster.
as an option for SELECT
New option is useful in case you run indexer --all and have not only plain indexes in the configuration file. Without ignore_non_plain=1 you'll get a warning and a respective exit code.
and enable visualization of full-text query plan execution. Useful for understanding complex queries.
cyrillic char period call snippets retain mode don't highlight
RTINDEX - GROUP N BY expression select = fatal crash
searchd status shows Segmentation fault when in cluster
'SHOW INDEX index.N SETTINGS' doesn't address chunks >9
Bug that crashes Manticore
Converter creates broken indexes
stopword_step=0 vs CALL SNIPPETS()
count distinct returns 0 at low max_matches on a local index
When using aggregation stored texts are not returned in hits
OPTIMIZE reduces disk chunks to a number of chunks ( default is 2* No. of cores) instead of a single one. The optimal number of chunks can be controlled by option.
NOT operator can be now used standalone. By default it is disabled since accidental single NOT queries can be slow. It can be enabled by setting new searchd directive to 0.
New setting sets how many threads a query can use. If the directive is not set, a query can use threads up to the value of . Per SELECT query the number of threads can be limited with overriding the global max_threads_per_query.
Percolate indexes can be now be imported with .
HTTP API /search receives basic support for / by new query node aggs.
outputs new metrics: killed_documents, killed_rate, disk_mapped_doclists, disk_mapped_cached_doclists, disk_mapped_hitlists and disk_mapped_cached_hitlists.
Crash of daemon at grouper at RT index with different chunks
Fastpath for empty remote docs
Expression stack frame detection runtime
Matching above 32 fields at percolate indexes
Replication listen ports range
Show create table on pq
HTTPS port behavior
Mixing docstore rows when replacing
Switch TFO unavailable message level to 'info'
Crash on strcmp invalid use
Adding index to cluster with system (stopwords) files
Merge indexes with large dictionaries; RT optimize of large disk chunks
Indextool can dump meta from current version
Issue in group order in GROUP N
Explicit flush for SphinxSE after handshake
Avoid copy of huge descriptions when not necessary
Negative time in show threads
Token filter plugin vs zero position deltas
Change 'FAIL' to 'WARNING' on multiple hits
This release took so long, because we were working hard on changing multitasking mode from threads to coroutines. It makes configuration simpler and queries parallelization much more straightforward: Manticore just uses given number of threads (see new setting ) and the new mode makes sure it's done in the most optimal way.
Changes in :
can be switched to global limits by limits_per_field=0 option (1 by default).
is now 0 by default for highlighting via HTTP JSON.
The same port for http, https and binary API (to accept connections from a remote Manticore instance). listen = *:mysql is still required for connections via mysql protocol. Manticore now detects automatically the type of client trying to connect to it except for MySQL (due to restrictions of the protocol).
In RT mode a field can now be at the same time - .
In it's called sql_field_string. Now it's available in for real-time indexes too. You can use it as shown in the example:
You can now .
Support of mysql client command.
can now replicate external files (stopwords, exceptions etc.).
Filter operator is now available via HTTP JSON interface.
in HTTP JSON.
in RT mode, i.e. can do ALTER ... rt_mem_limit=<new value>.
You can now use : chinese, japanese and korean.
now limits maximum thread stack, not initial.
cpustat, iostat, coredump can be changed during runtime with .
(i.e. without providing column list) previously expected exactly (query, tags) as the values. It's been changed to (id,query,tags,filters). The id can be set to 0 if you want it to be auto-generated.
is a new default in highlighting via HTTP JSON interface.
max_children is obsolete. Use to set the number of threads Manticore will use (set to the # of CPU cores by default).
queue_max_length is obsolete. Instead of that in case it's really needed use to fine-tune internal jobs queue size (unlimited by default).
read_timeout. Use instead which controls both reading and writing.
searchd memory leak
Tiny read out of bounds in snippets
Dangerous write into local variable for crash queries
Tiny memory leak of sorter in test 226
Huge memory leak in test 226
Cluster shows the nodes are in sync, but count(*) shows different numbers
Cosmetic: Duplicate and sometimes lost warning messages in the log
Cosmetic: (null) index name in log
Cannot retrieve more than 70M results
Can't insert PQ rules with no-columns syntax
Misleading error message when inserting a document to an index in a cluster
/json/replace and json/update return id in exponent form
Update json scalar properties and mva in the same query
hitless_words doesn't work in RT mode
ALTER RECONFIGURE in rt mode should be disallowed
rt_mem_limit gets reset to 128M after searchd restart
Failed to use U+code in RT mode
Failed to use wildcard at wordforms at RT mode
Fixed SHOW CREATE TABLE vs multiple wordform files
JSON query without "query" crashes searchd
Manticore couldn't index from mysql 8
HTTP /json/insert requires id
SHOW CREATE TABLE doesn't work for PQ
CREATE TABLE LIKE doesn't work properly for PQ
End of line in settings in show index status
Empty title in "highlight" in HTTP JSON response
CREATE TABLE LIKE infix error
RT crashes under load
Lost crash log on crash at RT disk chunk
Import table fails and closes the connection
ALTER reconfigure corrupts a PQ index
Searchd reload issues after change index type
Daemon crashes on import table with missed files
Crash on select using multiple indexes, group by and ranker = none
HIGHLIGHT() doesn't higlight in string attributes
FACET fails to sort on string attribute
Error in case of missing data dir
access_* are not supported in RT mode
Bad JSON objects in strings: 1. CALL PQ returns "Bad JSON objects in strings: 1" when the json is greater than some value.
RT-mode inconsistency. In some cases I can't drop the index since it's unknown and can't create it since the directory is not empty.
Crash on select
max_xmlpipe2_field = 2M returned warning on 2M field
Query conditions execution bug
Simple 2 terms search finds a document containing only one term
It was impossible in PQ to match a json with capital letters in keys
Indexer crashes on csv+docstore
using [null] in json attr in centos 7 causes corrupted inserted data
Major Records not being inserted, count() is random, "replace into" returns OK
Master-agent communication fails on Mac OS
Error when connecting to Manticore with Connector.Net/Mysql 8.0.19
Fixed escaping of \0 and optimized performance
Fixed count distinct vs json
Fixed drop table at other node failed
Fix crashes on tightly running call pq
fix RT index from old version fails to index data
fix crash on wrong field syntax in highlight()
fix crash of server on replicate RT index with docstore
fix crash on highlight to index with infix or prefix option and to index wo stored fields enabled
fix false error about empty docstore and dock-id lookup for empty index
fix #314 SQL insert command with trailing semicolon
removed warning on query word(s) mismatch
fix queries in snippets segmented via ICU
fix find/add race condition in docstore block cache
fix mem leak in docstore
fix #316 LAST_INSERT_ID returns empty on INSERT
fix #317 json/update HTTP endpoint to support array for MVA and object for JSON attribute
fix rash of indexer dumping rt without explicit id
added check of index schema for duplicate attributes #293
fix crash in hitless terms
fix loose docstore after ATTACH
fix docstore issue in distributed setup
replace FixedHash with OpenHash in sorter
fix attributes with duplicated names at index definition
fix html_strip in HIGHLIGHT()
fix passage macro in HIGHLIGHT()
fix double buffer issues when RT index creates small or large disk chunk
fix event deletion for kqueue
fix save of disk chunk for large value of rt_mem_limit of RT index
fix float overflow on indexing
fix insert document with negative ID into RT index fails with error now
fix crash of server on ranker fieldmask
fix crash on using query cache
fix crash on using RT index RAM segments with parallel inserts
fix crash on SELECT query over HTTP interface
fix RT index saves disk chunks but does not mark some documents deleted
fix crash on search of multi index or multi queries with dist_threads
fix crash on infix generation for long terms with wide utf8 codepoints
fix race at adding socket to IOCP
fix issue of bool queries vs json select list
fix indextool check to report wrong skiplist offset, check of doc2row lookup
fix indexer produces bad index with negative skiplist offset on large data
fix JSON converts only numeric to string and JSON string to numeric conversion at expressions
fix indextool exit with error code in case multiple commands set at command line
fix #275 binlog invalid state on error no space left on disk
fix #279 crash on IN filter to JSON attribute
fix #281 wrong pipe closing call
fix server hung at CALL PQ with recursive JSON attribute encoded as string
fix advancing beyond the end of the doclist in multiand node
fix retrieving of thread public info
fix docstore cache locks
fix crash on replace document in disk chunk of RT index
fix #269 LIMIT N OFFSET M
fix DELETE statements with id explicitly set or id list provided to skip search
fix wrong index after event removed at netloop at windowspoll poller
fix float roundup at JSON via HTTP
fix remote snippets to check empty path first; fixing windows tests
fix reload of config to work on windows same way as on linux
fix #194 PQ to work with morphology and stemmers
fix RT retired segments management
fixed replication of RT index into node where same RT index exists and has different path
fix flush rescheduling for indexes without activity
improve rescheduling of flushing RT/PQ indexes
fix #250 index_field_lengths index option for TSV and CSV piped sources
fix indextool wrong report for block index check on empty index
fix empty select list at Manticore SQL query log
fix indexer -h/--help response
fix defects in concat and group_concat
fix query uid at percolate index to be BIGINT attribute type
do not crash if failed to prealloc a new disk chunk
add missing timestamp data type to ALTER
fix crash of wrong mmap read
fix hash of clusters lock in replication
fix leak of providers in replication
fix #246 undefined sigmask in indexer
fix race in netloop reporting
zero gap for HA strategies rebalancer
allow attribute names starting with numbers in select list
fixed MVAs in UDFs, fixed MVA aliasing
fixed #187 crash when using query with SENTENCE
fixed #143 support () around MATCH()
fixed save of cluster state on ALTER cluster statement
fixed crash of server on ALTER index with blob attributes
fixed #196 filtering by id
discard searching on template indexes
fixed id column to have regular bigint type at SQL reply
fixed crash on many stars at select list for query into many distributed indexes
fixed large packet via Manticore SQL interface
fixed crash of server on RT optimize with MVA updated
fixed server crash on binlog removed due to RT index remove after config reload on SIGHUP
fixed mysql handshake auth plugin payloads
fixed phrase_boundary settings at RT index
fixed deadlock at ATTACH index to itself
fixed binlog saves empty meta after server crash
fixed crash of server due to string at sorter from RT index with disk chunks
fixed crash of server on match mode all and empty full text query
fixed crash on deleting of static string
fixed exit code when indextool failed with FATAL
fixed no matches for prefixes due to wrong exact form check
fixed reload of config settings for RT indexes
fixed crash of server on access of large JSON string
fixed PQ field at JSON document altered by index stripper causes wrong match from sibling field
fixed crash of server at parse JSON on RHEL7 builds
fixed crash of json unescaping when slash is on the edge
fixed option 'skip_empty' to skip empty docs and not warn they're not valid json
fixed output 8 digits on floats when 6 is not enough to be precise
fixed empty jsonobj creation
fixed empty mva outputs NULL instead of an empty string
fixed fail to build without pthread_getname_np
fixed crash on server shutdown with thread_pool workers
fixed csjon-related leak
fixed crash because of missed value in json
fixed save of empty meta for RT index
fixed lost form flag (exact) for sequence of lemmatizer
fixed string attrs > 4M use saturate instead of overflow
fixed crash of server on SIGHUP with disabled index
fixed server crash on simultaneous API session status commands
fixed crash of server at delete query to RT index with field filters
fixed crash of server at CALL PQ to distributed index with empty document
fixed cut Manticore SQL error message larger 512 chars
fixed crash on save percolate index without binlog
fixed http interface is not working in OSX
fixed indextool false error message on check of MVA
fixed write lock at FLUSH RTINDEX to not write lock whole index during save and on regular flush from rt_flush_period
fixed ALTER percolate index stuck waiting search load
fixed max_children to use default amount of thread_pool workers for value of 0
fixed error on indexing of data into index with index_token_filter plugin along with stopwords and stopword_step=0
fixed crash with absent lemmatizer_base when still using aot lemmatizers in index definitions
fixed crash of server on FACET with multiple attribute wide types
fixed implicit group by at main select list of FACET query
fixed crash on query with GROUP N BY
fixed deadlock on handling crash at memory operations
fixed indextool memory consumption during check
fixed gmock include not needed anymore as upstream resolve itself
fixed failed to build with libc without malloc_stats, malloc_trim
fixed special symbols inside words for CALL KEYWORDS result set
fixed broken CALL KEYWORDS to distributed index via API or to remote agent
fixed distributed index agent_query_timeout propagate to agents as max_query_time
fixed total documents counter at disk chunk got affected by OPTIMIZE command and breaks weight calculation
fixed multiple tail hits at RT index from blended
fixed deadlock at rotation
fixed crash or hfailure of rename on Windows
fixed crashes of server on 32-bit systems
fixed crash or hung of server on empty SNIPPET expression
fixed broken non progressive optimize and fixed progressive optimize to not create kill-list for oldest disk chunk
fixed queue_max_length bad reply for SQL and API at thread pool worker mode
fixed crash on adding full-scan query to PQ index with regexp or rlp options set
fixed crash when call one PQ after another
refactor AcquireAccum
fixed leak of memory after call pq
cosmetic refactor (c++11 style c-trs, defaults, nullptrs)
fixed memory leak on trying to insert duplicate into PQ index
fixed crash on JSON field IN with large values
fixed crash of server on CALL KEYWORDS statement to RT index with expansion limit set
fixed invalid filter at PQ matches query;
introduce small obj allocator for ptr attrs
refactor ISphFieldFilter to refcounted flavour
fixed ub/sigsegv when using strtod on non-terminated strings
fixed memory leak in json resultset processing
fixed read over the end of mem block applying attribute add
fixed refactor CSphDict for refcount flavour
fixed leak of AOT internal type outside
fixed memory leak tokenizer management
fixed memory leak in grouper
special free/copy for dynamic ptrs in matches (memory leak grouper)
fixed memory leak of dynamic strings for RT
refactor grouper
minor refactor (c++11 c-trs, some reformats)
refactor ISphMatchComparator to refcounted flavour
privatize cloner
simplify native little-endian for MVA_UPSIZE, DOCINFO2ID_T, DOCINFOSETID
add valgrind support to to ubertests
fixed crash because race of 'success' flag on connection
switch epoll to edge-triggered flavour
fixed IN statement in expression with formatting like at filter
fixed crash at RT index on commit of document with large docid
fixed argless options in indextool
fixed memory leak of expanded keyword
fixed memory leak of json grouper
fixed leak of global user vars
fixed leakage of dynamic strings on early rejected matches
fixed leakage on length()
fixed memory leak because strdup() in parser
fixed refactor expression parser to accurate follow refcounts
WITH RECONFIGURE
Manticore SQL command which can run various subcommands
- SHA1 hash of password needed to invoke shutdown using DEBUG command
removed wlock at optimize
fixed wlock at reload index settings
fixed memory leak on query with JSON filter
fixed empty documents at PQ result set
fixed confusion of tasks due to removed one
fixed wrong remote host counting
fixed memory leak of parsed agent descriptors
fixed leak in search
cosmetic changes on explicit/inline c-trs, override/final usage
fixed leak of json in local/remote schema
fixed leak of json sorting col expr in local/remote schema
fixed leak of const alias
fixed leak of preread thread
fixed stuck on exit because of stucked wait in netloop
fixed stuck of 'ping' behaviour on change HA agent to usual host
separate gc for dashboard storage
fixed ref-counted ptr fix
fixed indextool crash on unexistent index
fixed output name of exceeding attr/field in xmlpipe indexing
fixed default indexer's value if no indexer section in config
fixed wrong embedded stopwords in disk chunk by RT index after server restart
fixed skip phantom (already closed, but not finally deleted from the poller) connections
fixed blended (orphaned) network tasks
fixed crash on read action after write
fixed searchd crashes when running tests on windows
fixed handle EINPROGRESS code on usual connect()
fixed connection timeouts when working with TFO
fixed loose of query string
fixed empty info at SHOW THREADS statement
fixed crash on matching with NOTNEAR operator
fixed error message on bad filter to PQ delete
now outputs to server version and mysql_version_string
fixed added not equal to tags of PQ
fixed added document id field to JSON document CALL PQ statement
fixed flush statement handlers to PQ index
fixed PQ filtering on JSON and string attributes
fixed parsing of empty JSON string
fixed crash at multi-query with OR filters
fixed indextool to use config common section (lemmatizer_base option) for commands (dumpheader)
fixed empty string at result set and filter
fixed negative document id values
fixed word clip length for very long words indexed
fixed matching multiple documents of wildcard queries at PQ
MySQL FEDERATED engine
- enable TCP Fast Open connections for all listeners
fixed invalid query cache entries for RT index;
fixed index settings got lost next after seamless rotation
fixed fixed infix vs prefix length set; added warning on unsupportedinfix length
fixed RT indexes auto-flush order
fixed result set schema issues for index with multiple attributes and queries to multiple indexes
fixed some hits got lost at batch insert with document duplicates
fixed optimize failed to merge disk chunks of RT index with large documents count
fixed log expand_keywords option into Manticore SQL query log
fixed HTTP interface to correctly process query with large size
fixed crash of server on DELETE to RT index with index_field_lengths enable
fixed cpustats searchd cli option to work with unsupported systems
fixed utf8 substring matching with min lengths defined
improved performance in case of using NOT operator and for batched documents.
can use multiple threads depending on
can accept 'start','exact' (where 'star,exact' has same effect as '1')
ranged-main-query for which uses the ranged query defined by sql_query_range
fixed crash on searching ram segments; deadlock on save disk chunk with double buffer; deadlock on save disk chunk during optimize
fixed indexer crash on xml embedded schema with empty attribute name
fixed erroneous unlinking of not-owned pid-file
fixed orphaned fifos sometimes left in temp folder
fixed empty FACET result set with wrong NULL row
fixed broken index lock when running server as windows service
fixed wrong iconv libs on mac os
fixed wrong count(*)
in case of agents with mirrors gives the value of retries per mirror instead of per agent, the total retries per agent being agent_retry_count*mirrors.
can now be specified per index, overriding global value. An alias is added.
Percolate Queries are now in HTTP JSON API at .
support for Real-Time indexes
fixed ranged-main-query to correctly work with sql_range_step when used at MVA field
fixed issue with blackhole system loop hung and blackhole agents seems disconnected
fixed query id to be consistent, fixed duplicated id for stored queries
fixed server crash on shutdown from various states
timeouts on long queries
refactored master-agent network polling on kqueue-based systems (Mac OS X, BSD).
: JSON queries can now do equality on attributes, MVA and JSON attributes can be used in inserts and updates, updates and deletes via JSON API can be performed on distributed indexes
. A new index directive morphology_skip_fields allows defining a list of fields for which morphology does not apply.
fixed crash on debug build of server (and m.b. UB on release) when built with rlp
fixed RT index optimize with progressive option enabled that merges kill-lists with wrong order
minor crash on mac
JSON queries on . Supported search, insert, update, delete, replace operations. Data manipulation commands can be also bulked, also there are some limitations currently as MVA and JSON attributes can't be used for inserts, replaces or updates.
command
command
can show progress of optimize, rotation or flushes.
filter regression with block index
rename PAGE_SIZE -> ARENA_PAGE_SIZE for compatibility with musl
disable googletests for cmake < 3.1.0
failed to bind socket on server restart
fixed crash of server on shutdown
fixed show threads for system blackhole thread
Refactored config check of iconv, fixes building on FreeBSD and Darwin
available on distributed indexes
for custom log files permissions
can now affect full-scans
added , and for network thread fine tuning (in case of workers=thread_pool)
shows a multiplier row when multi-query optimization is used
fixed SHOW COLLATION statement that breaks java connector
fixed crashes on processing distributed indexes; added locks to distributed index hash; removed move and copy operators from agent
fixed crashes on processing distributed indexes due to parallel reconnects
fixed crash at crash handler on store query to server log
fixed a crash with pooled attributes in multiqueries
fixed reduced core size by prevent index pages got included into core file
fixed searchd crashes on startup when invalid agents are specified
fixed indexer reports error in sql_query_killlist query
fixed fold_lemmas=1 vs hit count
fixed inconsistent behavior of html_strip
fixed optimize rt index loose new settings; fixed optimize with sync option lock leaks;
fixed processing erroneous multiqueries
fixed result set depends on multi-query order
fixed server crash on multi-query with bad query
fixed shared to exclusive lock
fixed server crash for query without indexes
fixed dead lock of server
