Commit graph

12 commits

Author SHA1 Message Date
Sarjuuk
5a230daad6 Search/Indexing
* move fulltext indizes for tables /w ~10k+ rows to separate tables
   > sounds have ~12k but names are effectively incompatible with FTI
 * normalize searchable strings to catch edgecases
   > preparse spell descriptions + buffs
   > move effect text cols of items to new table
 * also fix extended search of creatures, spells & quests
2026-03-03 21:02:33 +01:00
Sarjuuk
2161a7b846 Search/Fixup
* dashes also split words into multiple tokens, so they are now
   stripped from search entirely (see: 'weather-beaten')
 * only search for the full search input if the tokenizer didn't yield
   any fulltext tokens. This should fix search performance after 00f048d3ae
 * fixes #497
2026-02-27 12:02:38 +01:00
Sarjuuk
69df50619a DB/Dependency
* remove unmaintained DbSimple
 * add package db/dibi as substitute db abstraction
2026-02-26 16:26:02 +01:00
Sarjuuk
2216b664fa DBTypes/Filters
* filters should test string input during initialization, so errors
   can be passed on to the PageTemplate.
 * have strings be compared for identity by default and (NOT) LIKE as
   optional operator
 * note: if a string criterium gets processed via callback function
   it will not get prechecked
2026-02-12 14:47:24 +01:00
Sarjuuk
00f048d3ae Search/Fixup
* readd regular indizes for name cols. There are cases where entities
   are named in a way that does not work with FT indizes.
   ex. "XT:9" is two tokens "XT", "9" which are too short to be indexed.
 * additionally to FT search also exact match col to search string.
2026-02-12 00:27:49 +01:00
Sarjuuk
4c598972cc Search/Misc
* why loop when regex does trick
 * further unify variable naming and layout
2026-01-19 20:15:11 +01:00
Sarjuuk
53374972ef Search/Fixup
* strip leading/trailing dashes from search tokens.
   Casues FT syntax errors when combined with our own modifiers
2026-01-19 19:31:35 +01:00
Sarjuuk
6df9145446 Search/Fixup
* do not use stopwords in fulltext search
   apparentyl there is a bug in innodb where including stopwords in a search causes the lookup to fail entirely.
   e.g. innodb_ft_enable_stopword must be disabled when the index is edited (rows are added/removed)
 * don't create a MATCH AGAINST search from empty search strings after sanitization
 * drop fulltext indizes for locale zhCN
   logographic languages need special treatment, which handling may differ by db provider
 * use LIKE search by default for locale zhCN. Added config option to use fulltext if supported by db.
2026-01-17 23:07:44 +01:00
Sarjuuk
7616ec25fc DB/Search
* add more indizes to large tables for cols used in lookups
 * drop multi-column indizes on spell as they are not utilized by mysql
 * add and use fulltext indizes for names of items, spells, quests, creatures & objects
   could add more, but is it really necessary?
 * limitations
   - still need a solution for race/class/spellFamily masks cols that are used as such
   - fulltext indizes in boolean mode cant partial match the end of a word.
     reverse name cols and search and match back to front like that..? blows up db size even more though
     (+trike* : "stormstrike" => +ekirt* : "ekirtsmrots")
2026-01-16 16:56:24 +01:00
Sarjuuk
c44bf4f575 Config/Misc
* hardcode sql limits within their respective components instead of
   having them configurable
 * creating a new DBTypeList defaults it to being unlimited
   * SQL_LIMIT_NONE (0) > removed as it only existed for consistency
   * SQL_LIMIT_DEFAULT (300) > frontend / Listview::DEFAULT_SIZE
   * SQL_LIMIT_SEARCH (500) > Search::DEFAULT_MAX_RESULTS
   * SQL_LIMIT_QUICKSEARCH (10) > Search::SUGGESTIONS_MAX_RESULTS
2025-12-30 03:20:09 +01:00
Sarjuuk
e37620c01b Search/Fixup
* fix pruning empty tokens from search
2025-10-06 17:06:43 +02:00
Sarjuuk
1f5152c871 Template/Update (Part 4)
* convert search into separate endpoints
 * move shared functionalty to components
 * NOTE: acceptance of opensearch has waned over the last decade and
         the script should be updated
2025-09-25 15:55:37 +02:00