BACKGROUND IMAGE: iSTOCK/GETTY IMAGES
Microsoft SQL Server is a relational database management system (RDBMS) that supports a wide variety of transaction processing, business intelligence and analytics applications in corporate IT environments. Microsoft SQL Server is one of the three market-leading database technologies, along with Oracle Database and IBM's DB2.
Like other RDBMS software, Microsoft SQL Server is built on top of SQL, a standardized programming language that database administrators (DBAs) and other IT professionals use to manage databases and query the data they contain. SQL Server is tied to Transact-SQL (T-SQL), an implementation of SQL from Microsoft that adds a set of proprietary programming extensions to the standard language.
Inside SQL Server's architecture—How SQL Server works
Like other RDBMS technologies, SQL Server is primarily built around a row-based table structure that connects related data elements in different tables to one another, avoiding the need to redundantly store data in multiple places within a database. The relational model also provides referential integrity and other integrity constraints to maintain data accuracy. Those checks are part of a broader adherence to the principles of atomicity, consistency, isolation and durability, collectively known as the ACID properties, and are designed to guarantee that database transactions are processed reliably.
The core component of Microsoft SQL Server is the SQL Server Database Engine, which controls data storage, processing and security. It includes a relational engine that processes commands and queries and a storage engine that manages database files, tables, pages, indexes, data buffers and transactions. Stored procedures, triggers, views and other database objects are also created and executed by the Database Engine.
Sitting beneath the Database Engine is the SQL Server Operating System, or SQLOS. SQLOS handles lower-level functions, such as memory and I/O management, job scheduling and locking of data to avoid conflicting updates. A network interface layer sits above the Database Engine and uses Microsoft's Tabular Data Stream protocol to facilitate request and response interactions with database servers. And at the user level, SQL Server DBAs and developers write T-SQL statements to build and modify database structures, manipulate data, implement security protections and back up databases, among other tasks.
SQL Server services, tools and editions
Microsoft also bundles a variety of data management, business intelligence (BI) and analytics tools with SQL Server. In addition to the R Services and now Machine Learning Services technology that first appeared in SQL Server 2016, the data analysis offerings include SQL Server Analysis Services, an analytical engine that processes data for use in BI and data visualization applications, and SQL Server Reporting Services, which supports the creation and delivery of BI reports.
On the data management side, Microsoft SQL Server includes SQL Server Integration Services, SQL Server Data Quality Services and SQL Server Master Data Services. Also bundled with the DBMS are two sets of tools for DBAs and developers: SQL Server Data Tools, for use in developing databases, and SQL Server Management Studio, for use in deploying, monitoring and managing databases.
SQL Server editions:
Microsoft offers SQL Server in four primary editions that provide different levels of the bundled services. Two are available free of charge: a full-featured Developer edition for use in database development and testing, and an Express edition that can be used to run small databases with up to 10 GB of disk storage capacity. For larger applications, Microsoft sells an Enterprise edition that includes all of SQL Server's features, as well as a Standard one with a partial feature set and limits on the number of processor cores and memory sizes that users can configure in their database servers.
Other editions include a Standard version, Web version and a Business Intelligence version. The Business Intelligence version provides Self Service and Corporate Business Intelligence tools with the Standard Edition capabilities.
However, when SQL Server 2016 Service Pack 1 (SP1) was released in late 2016, Microsoft made some of the features previously limited to the Enterprise edition available as part of the Standard and Express ones. That included In-Memory OLTP, PolyBase, columnstore indexes, and partitioning, data compression and change data capture capabilities for data warehouses, as well as several security features. In addition, the company implemented a consistent programming model across the different editions with SQL Server 2016 SP1, making it easier to scale up applications from one edition to another.
The original SQL Server code was developed in the 1980s by the former Sybase Inc., which is now owned by SAP. Sybase initially built the software to run on Unix systems and minicomputer platforms. It, Microsoft and Ashton-Tate Corp., then the leading vendor of PC databases, teamed up to produce the first version of what became Microsoft SQL Server, designed for the OS/2 operating system and released in 1989.
Ashton-Tate stepped away after that, but Microsoft and Sybase continued their partnership until 1994, when Microsoft took over all development and marketing of SQL Server for its own operating systems. The year before, with the Sybase relationship starting to unravel, Microsoft had also made the software available on the newly released Windows NT after modifying the 16-bit OS/2 code base to create a 32-bit implementation with added features; it focused on the Windows code going forward. In 1996, Sybase renamed its version Adaptive Server Enterprise, leaving the SQL Server name to Microsoft.
Versions of SQL Server
Between 1995 and 2016, Microsoft released ten versions of SQL Server. Early versions were aimed primarily at departmental and workgroup applications, but Microsoft expanded SQL Server's capabilities in subsequent ones, turning it into an enterprise-class relational DBMS that could compete with Oracle Database, DB2 and other rival platforms for high-end database uses. Over the years, Microsoft has also incorporated various data management and data analytics tools into SQL Server, as well as functionality to support new technologies that emerged, including the web, cloud computing and mobile devices.
SQL Server 2012:
Prior versions included SQL Server 2005, SQL Server 2008 and SQL Server 2008 R2, which was considered a major release despite the follow-up sound of its name. Next to come were SQL Server 2012 and SQL Server 2014. SQL Server 2012 offered new features, such as columnstore indexes, which can be used to store data in a column-based format for data warehousing and analytics applications, and AlwaysOn Availability Groups, a high availability and disaster recovery technology. (Microsoft changed the spelling of the latter's name to Always On when it released SQL Server 2016.)
SQL Server 2014:
SQL Server 2014 added In-Memory OLTP, which lets users run online transaction processing (OLTP) applications against data stored in memory-optimized tables instead of standard disk-based ones. Another new feature in SQL Server 2014 was the buffer pool extension, which integrates SQL Server's buffer pool memory cache with a solid-state drive -- another feature designed to boost I/O throughput by offloading data from conventional hard disks.
Microsoft SQL Server 2016:
Microsoft SQL Server 2016, which became generally available in June 2016, was developed as part of a "mobile first, cloud first" technology strategy adopted by Microsoft two years earlier. Among other things, SQL Server 2016 added new features for performance tuning, real-time operational analytics, and data visualization and reporting on mobile devices, plus hybrid cloud support that lets DBAs run databases on a combination of on-premises systems and public cloud services to reduce IT costs. For example, a SQL Server Stretch Database technology moves infrequently accessed data from on-premises storage devices to the Microsoft Azure cloud, while keeping the data available for querying, if needed.
SQL Server 2016 also increased support for big data analytics and other advanced analytics applications through SQL Server R Services, which enables the DBMS to run analytics applications written in the open source R programming language, and PolyBase, a technology that lets SQL Server users access data stored in Hadoop clusters or Azure blob storage for analysis. Also, SQL Server 2016 was the first version of the DBMS to run exclusively on 64-bit servers based on x64 microprocessors. And it added the ability to run SQL Server in Docker containers, a virtualization technology that isolates applications from each other on a shared operating system.
SQL Server 2017:
Microsoft SQL Server ran exclusively on Windows for more than 20 years. But, in 2016, Microsoft said it planned to also make the DBMS available on Linux, starting with a new version released as a community technology preview that November and initially dubbed SQL Server vNext; later, the update was formally named SQL Server 2017, and it became generally available in October of that year.
The support for running SQL Server on Linux moved the database platform onto an open source operating system commonly found in enterprises, giving Microsoft potential inroads with customers that don't use Windows or have mixed server environments. SQL Server 2017 also expanded the Docker support added for Windows systems in the previous release to include Linux-based containers.
Another notable feature in SQL Server 2017 is support for the Python programming language, an open source language that is widely used in analytics applications. With its addition, SQL Server R Services was renamed Machine Learning Services (In-Database) and expanded to run both R and Python applications. Initially, the machine learning toolkit and a variety of other features are only available in the Windows version of the database software, with a more limited feature set supported on Linux.
SQL Server 2019:
SQL Server 2019 is slated to release in late 2019; however, a preview is available for download. SQL Server 2019 CTP 3.0 is currently the latest public version available for preview— only as an Evaluation Edition. SQL Server 2019 was shown in September 2018, introducing new features and tweaks focusing on performance, security and increasing work volume data.
SQL Server 2019 allows users to join SQL Server, HDFS, and Spark containers together using a new Big Data Cluster feature. SQL Server 2019 also introduces columnstore index builds, rebuilds and static data masking. Accelerated Data Recovery is also new, which performs and undoes a redo phase in the oldest page log sequence number. As an example, this is done in the case scenario where the user closes an application which was running for an extended period of time, so the user does not have to wait long for the application to close.
Always On Availability Groups, available in SQL Server 2012, has been changed to simplify administration of availability groups. This adds support to MSDB and Master system databases. Other changes to features include the expansion to operations users can perform with Always Encrypted data; additional Polybase connectors for SQL Server, Oracle, MongoDB and Teradata; additional persistent memory options for storage; and improvements on Query Processing.
Security features in SQL Server
The advanced security features supported in all editions of Microsoft SQL Server starting with SQL Server 2016 SP1 include three technologies added to the 2016 release: Always Encrypted, which lets user update encrypted data without having to decrypt it first; row-level security, which enables data access to be controlled at the row level in database tables; and dynamic data masking, which automatically hides elements of sensitive data from users without full access privileges.
Other notable SQL Server security features include transparent data encryption, which encrypts data files in databases, and fine-grained auditing, which collects detailed information on database usage for reporting on regulatory compliance. Microsoft also supports the Transport Layer Security protocol for securing communications between SQL Server clients and database servers.
Most of those tools and the other features in Microsoft SQL Server are also supported in Azure SQL Database, a cloud database service built on the SQL Server Database Engine. Alternatively, users can run SQL Server directly on Azure, via a technology called SQL Server on Azure Virtual Machines; it configures the DBMS in Windows Server virtual machines running on Azure. The VM offering is optimized for migrating or extending on-premises SQL Server applications to the cloud, while Azure SQL Database is designed for use in new cloud-based applications.
In the cloud, Microsoft also offers Azure SQL Data Warehouse, a data warehousing service based on a massively parallel processing (MPP) implementation of SQL Server. The MPP version, originally a stand-alone product called SQL Server Parallel Data Warehouse, is also available for on-premises uses as part of the Microsoft Analytics Platform System, which combines it with PolyBase and other big data technologies.