Home > SQL Server Tips > Database Administration > Using full-text search for symbols in SQL Server
SQL Server Tips:
EMAIL THIS
 TIPS & NEWSLETTERS TOPICS 

DATABASE ADMINISTRATION

Using full-text search for symbols in SQL Server


Michelle Gutzait, Contributor
08.06.2008
Rating: -4.50- (out of 5)


Expert advice on database administration
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


Problem:

I have a table with a full-text-search index defined on it. I would like to search an exact SQL expression including symbols such as ! " @, but the FTS is ignoring these symbols by default. For example, if I search for the exact phrase: "SQL Server!" using CONTAINSTABLE, I get all expressions containing "SQL Server" only – the exclamation point is ignored in the search.

How can I work around this issue?

Explanation:

FTS differentiates between a symbol defined as a wildcard and the other symbols. Here are a few symbols used in FTS as wildcards:

[TABLE]

The problem is that full-text Search in SQL Server does not index symbols, such as ^, +, ! and @, and therefore these symbols are ignored when part of an FTS query. The wildcard symbols can be treated as regular symbols if the escape character "\" appears before.

Example #1: Wildcards

The following query uses the escape character "\" to indicate that the following wildcard symbol should be treated as a simple character ("+"):

SELECT ID, Title
FROM dbo.FTS_Table
INNER JOIN CONTAINSTABLE(dbo.FTS_Table, (title),
'"New World \+"', 10) AS KEY_TBL ON FTS_Table.ID = KEY_TBL.[KEY]

Returns:

[TABLE]

Example #2: Non-wildcard symbols

The following query returns no results:

SELECT ID, Title
FROM dbo.FTS_Table
INNER JOIN CONTAINSTABLE(dbo.FTS_Table, (title),
'"@"', 10) AS KEY_TBL ON FTS_Table.ID = KEY_TBL.[KEY]

This is because FTS does not index the "@" character, while the following query:

select * from dbo.FTS_Table where Title like '%@%'

Returns:

[TABLE]

That's because the LIKE clause searches the character in the string without using the FTS index.

Workaro


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


RELATED CONTENT
Microsoft SQL Server Performance Monitoring and Tuning
Performance implications of transaction log autogrowth in SQL Server
The short course on how SQL Server really works
Determining the source of full transaction logs in SQL Server
Improving SQL Server full-text search performance
New GROUP BY option provides better data control in SQL Server 2008
Microsoft SQL Server 2008 Resource Governor primer
Examining data files when SQL Server tempdb is full
Testing transaction log autogrowth behavior in SQL Server
Meeting business needs with SQL Server full-text search
Using dynamic management views to improve SQL Server index effectiveness

Database Administration
Top load balancing methods for SQL Server
Performance implications of transaction log autogrowth in SQL Server
The keys to database backup protection for SQL Server
Understanding transparent data encryption in SQL Server 2008
Working with sparse columns in SQL Server 2008
Determining the source of full transaction logs in SQL Server
Implementing SQL Server 2008 FILESTREAM functionality
Improving SQL Server full-text search performance
Using the OPENROWSET function in SQL Server
New replication features in SQL Server 2008 and what they mean to you

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
contiguity  (SearchSQLServer.com)
contiguous  (SearchSQLServer.com)
drilldown  (SearchSQLServer.com)
hashing  (SearchSQLServer.com)
hybrid online analytical processing  (SearchSQLServer.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary


und

It is possible to add a WHERE clause with LIKE as follows:

SELECT ID, Title
FROM dbo.FTS_Table
INNER JOIN CONTAINSTABLE(dbo.FTS_Table, (title),
'"New World!"', 10) AS KEY_TBL ON FTS_Table.ID = KEY_TBL.[KEY]
WHERE title LIKE '%New World !%'

Results:

[TABLE]

Performance

When looking at the performance implications of the WHERE clause, it shows no difference between the two methods:

[IMAGE]
Figure 1: Query execution plan shows no performance difference using the WHERE clause in FTS.

Statistics IO:

(10 row(s) affected)
Table 'FTS_Table'. Scan count 0, logical reads 30, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

(1 row(s) affected)

(0 row(s) affected)
Table 'FTS_Table'. Scan count 0, logical reads 30, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

(1 row(s) affected)

SQL Profiler:

[IMAGE]
Figure 3: SQL Profiler shows no performance impact when using the WHERE clause in FTS. (Click on image for enlarged view.)

Ranking symbols

There are cases when you must return all phrases containing the exact phrase -- no matter if there are symbols or not -- but you can rank the exact phrase containing the symbol higher. For that matter, a CASE can be added to the query and the results can be ordered by it:

SELECT ID, Title,
CASE WHEN title like '%New World !%' then 1
ELSE 2
END AS ord

FROM dbo.FTS_Table
INNER JOIN CONTAINSTABLE(dbo. FTS_Table, (title), '"New World !"', 10)
AS KEY_TBL ON FTS_Table.ID = KEY_TBL.[KEY]
ORDER BY 3

Results:

[TABLE]

If the column "ord" should not be returned, a Common Table Expression (CTE) can be used:

WITH fts_example (id, title,ord) AS
(SELECT ID, Title, CASE WHEN title like '%New World !%' then 1
ELSE 2
END
FROM dbo. FTS_Table
INNER JOIN CONTAINSTABLE(dbo. FTS_Table, (title), '"New World!"', 10)
AS KEY_TBL ON FTS_Table.ID = KEY_TBL.[KEY])
select ID, Title from fts_example
order by ord

In comparing the last two queries, there's no difference in the execution plan, statistics IO or SQL Profiler.

If you have to query exact phrases including symbols using full-text Search in SQL Server, you need to add your own logic, such as a WHERE clause with LIKE containing your symbol, a CASE in the SELECT list, etc. Performance is not dramatically decreased -- and it's often not decreased at all.


[TABLE]


Rate this Tip
To rate tips, you must be a member of SearchSQLServer.com.
Register now to start rating these tips. Log in if you are already a member.


Submit a Tip




DISCLAIMER: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.



SQL Server Development - .NET, C#, T-SQL, Visual Basic
HomeNewsTopicsITKnowledge ExchangeTipsAsk the ExpertsMultimediaWhite PapersIT Downloads
About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Site Map




All Rights Reserved, Copyright 2005 - 2009, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts