ObjectPascal: Creating a REST Server
In today's post, I will be going through the creation of a basic REST Server created using the SQLDBRestBridge components. I am also planning on putting up an example website in the coming days to showcase various REST Server examples along with some pas2js web applications to use with it.
Why Use REST?
For those that might be wondering, why REST? For me the main reason is to use the pas2js Dataset components much easier. If you plan on building front-ends that need to load large datasets to process, then REST can be a decent fit. It can also be used to easily fetch single records, as well as performing various filters. I highly recommend checking out that link above to the SQLDBRestBridge page, as it explains a lot of what you can do with a REST Server.
It took me a bit of time to properly research and understand how to make a properly functioning REST Server using these components. Unfortunately, the documentation can be sparse in some places, and difficult to understand in others. Some concepts were a bit more difficult for me to grasp at first, but once I figured it all out, I was able to create a very power REST server this morning which is working with a pas2js front-end, and able to load in a dataset, change between records, create new records, and update records. I will only be going through the REST Server functionality in this post, and will not be writing the code for the pas2js front-end code. I will have a separate post to write about the front-end, as there is quite a lot to go through in terms of just the REST Server itself, that I thought it would be best to just dedicate a full post to it, rather than jumping between two projects.
Creating a New REST Server Project
Let's begin. Open up Lazarus, and you should have the lazsqldbrest, sqldbrestschemadesigner, weblaz, and sqldblaz packages installed to create this example project. Create a new Project, select the FastCGI Application. It will automatically create a new Data Module for you, we don't need this Data Module, and so you can remove it from your project. Close the form window, close the file in your source editor, and remove it from the Project Inspector.
The Data Module we are looking for however is in File->New..., under the Units section, locate SQLDB REST Bridge Module and add the unit to your project. In the Properties, select a Name for the Data Module.
In the main project file, you should uncomment the port setting and choose something that you can reference from your FastCGI web server.
The REST Dispatcher
The first component we need is the REST Dispatcher, this can be located in the component palette under the fpWeb tab as TSQLDBRestDispatcher. Click the component, then click an empty space on your Data Module form window to add it. In the Properties choose a Name for it. Then click into the main form and select it as the Dispatcher in the Data Module Properties.
Most of the properties on the TSQLDBRestDispatcher are fine for this example, however, you should set the BasePath to /api/.
I will not be providing any database schemas in this example, the assumption is that you already have some database tables that you'd like to expose over REST, or at the very least, can create a few example tables of your own. However, I will be providing SQL for an authentication table to ease anyone who might get frustrated at getting the authentication to work. This SQL will come later in the post in the section about adding authentication.
With that out of the way, we need to create at least one database connection for our REST Server to use. This is done by clicking on the Connections property, then pressing the button on the far-right of the field. This will open a dialog box where you can manage the database connections, choose Add. The new connection will populate in the Properties. Choose a Name for your connection. Fill out all the other details, including the password for the connection. The connection needs to be valid for the Schema Editor to function the way it needs to for our example here.
Once the connection properties have all been filled in, click onto the Dispatcher component in the Data Module form window to put the properties back to it. In the DefaultConnection dropdown, choose your newly created connection.
Defining our REST Schema
This next part will assume that all the tables you want to expose via this REST Server are a part of the connection you added to the dispatcher above.
In the component palette, under the fpWeb tab, locate TSQLDBRestSchema. Click onto the component, and then onto an empty space in your Data Module form window to add the component. Give it a Name in the properties, and enter in the connection name into the ConnectionName property field.
Okay, now we are ready to create the REST Schema. Right-click on the TSQLDBRestSchema component in the Data Module form window, and select Edit Schema. This will open a new dialog called the Schema Designer. As your database tables will be different than mine or the next person's, I will not overly explain this editor. Click onto the link I provided there to learn how to add your database tables. You should see the connection you added in the leftmost pane. You can drag and drop the tables from this connection into the next pane under Available Resources to add them to your schema with some defaults. Overall the Schema Designer should be easy enough for most developers to understand. Once your REST Schema has been created, press Ok and return back to this post.
There is a bug with the Schema Designer, at least with my version, in that when I click onto the table under Available Resources, I get an Access Violation error. I've found that this error is safe to just ignore by clicking on the Ok button and not clicking in that section of the Schema Designer. This information can be edited on the actual resource in the IDE. However, this also brings me to another bug, after closing the Schema Designer, it does not update the resources in the IDE. So, before continuing with this guide, close and re-open your project, and it will correctly populate the resources. You should also do everything you need to in the Schema Editor due to yet another bug, where if you update the resource in the IDE, and then edit the schema via the Schema Designer, all those changes are reverted. This includes the event handlers, which can be a bit annoying to say the least.
Setting up the REST Resources
Once the REST Schema has been created, and you closed and re-opened your project, you should see each of the tables listed as Resources under the Resources property for the Schema component. It will say how many resources you have, if you look above this property area into the component tree, you should see the tables under Resources there. It is the resources in the component tree that we are most interested in. Click onto one of your newly created resources, and let's begin configuring one.
These resource properties should be set to how you want your REST Server to handle each of these resources. For example, the first property Allowed Operations can be altered so that you may not even need authentication if you want a purely read-only REST server. To have a read-only resource, uncheck all the options here except for roGet, roHead, and roOptions. If you are not familiar with REST, I'll explain the other options, roDelete should explain itself. roPost is for adding new records. roPut is for updating an existing record. These allow options allow you to fine-tune the default access for a resource. You will generally not need to change any of these other properties. However, the SQL ones might be of interest, however, they are auto-generated if they are left blank, which is the recommended usage.
The most interesting part about a REST Resource is under the Events tab. I will try to explain how you can use each of these events to really fine-tune the REST Server and how it provides resources:
OnAllowedOperations: If you need to override the defaults set in the properties for some requests, this is how you do it.OnAllowRecord: This can be used to tweak if a record is provided in the resource or not. Very useful when combined with authentication, but can also be used for example to check if a document is published or not.OnCheckParams: This one can be used to check or alter parameters before it performs the database query. For example, on an authenticated server, I have used this to enforce a uid field on a table when a POST is done so that the data can be owned and accessed by that user.OnGetDataset: This one really confused me at first, but it is solely meant to provide custom datasets unrelated to the actual resource, you should never need to use this.OnResourceAllowed: Paired with authentication can be used to limit what say a guest user has access to. We'll talk more about guests in the upcoming authentication section.
If you don't care about authentication on your REST Server, then that is really all you need. Just run your REST Server, and it should be available via /REST/api/ for REST clients to consume.
Adding Authentication
Authentication can be added in two different ways, if you've explored the properties on the Dispatcher, you've probably noticed Authenticator and the event OnBasicAuthentication. The latter can be used if you want to roll your own authentication either with hardcoded information, or from a custom external datasource. The one we will be using in this guide however, is the former.
In the component palette, under the fpWeb tab, locate the TRestBasicAuthenticator and place the component into your Data Module form window to add it. Choose a Name for it, and then select it from the Authenticator dropdown in the Dispatcher properties. Also, in the Dispatcher properties, locate the property DispatchOptions, expand it and select rdoAccessCheckNeedsDB. This option is very important when using this dispatcher component, as our authentication data is stored in the database.
Click onto the newly added Authenticator component to view it's properties. The first property, AuthConnection is only used if you are using a different connection than the rest of the REST Server is using. The DefaultUsername and other similar properties are used to enable guest access. The username here cannot be one from your authentication table, and the DefaultUserID property can be used in the REST Server code to determine if the current user is a guest or not. However, do note, that this guest access isn't automatic if no authentication information is passed, which is why a DefaultPassword is also here. If authentication is enabled, it is required, and this fake account here is a provision to allow clients you create to access the REST Server using guest access, but it still needs to be entered in from a client to be used.
CREATE TABLE IF NOT EXISTS public.passwd
(
id integer NOT NULL,
username character varying(40) COLLATE pg_catalog."default" NOT NULL,
password character varying(60) COLLATE pg_catalog."default" NOT NULL,
CONSTRAINT passwd_pkey PRIMARY KEY (id)
)
TABLESPACE pg_default;
ALTER TABLE IF EXISTS public.passwd
OWNER to postgres;
CREATE SEQUENCE IF NOT EXISTS public.passwd_id_seq
INCREMENT 1
START 1
MINVALUE 1
MAXVALUE 2147483647
CACHE 1;
ALTER SEQUENCE public.passwd_id_seq
OWNER TO postgres;
This is a very basic PostgreSQL table for our authentication to use. Do note that we do not use the IDENTITY in the database server, but instead, we create our own sequence which will need to be set in the REST Schema. At least for PostgreSQL, each table you wish to expose with write access will need to have the IDs generated in this way, do not use the IDENTITY ALWAYS constraint on tables you wish to add records to. I learned this the hard way the other day, as I was stuck and stumped on why I couldn't add new records to my REST Server. Creating all your tables as I mentioned above will save you a lot of headaches. Be sure to add this sequence to the Sequence Name in the Schema Designer as well.
SELECT id FROM public.passwd WHERE username=:UserName AND password=crypt(:Password,(SELECT password FROM public.passwd WHERE username=:UserName))
This is an example to use in the AuthenticateUserSQL property for the Authenticator component. To add a new user for example, you can use an SQL statement like this:
INSERT INTO public.passwd (id,username,password) VALUES (nextval('passwd_id_seq'),:Username,crypt(:Password, gen_salt('bf')))
It can be possible to add new users via your REST API, but you will first need to specify a user whom will be given such abilities. You would need to check the UserID via the resource's OnAllowedOperations with code like this for example:
procedure TRESTModule.RESTSchemaResources1AllowedOperations(aSender: TObject;
aContext: TBaseRestContext; var aOperations: TRestOperations);
begin
if aContext.UserID = '1' then { Replace with your ID from the DB. }
aOperations:=[roPost, roGet, roHead, roOptions];
end;
Yes, the user ID is hardcoded, but unfortunately, I don't currently know a better way. Look forward to a future post where I do end up researching this and have a more elegant solution to offer.
However, if you do choose to expose your authentication table, which you actually don't need to, I'd recommend placing this event onto the resource's OnAllowRecord event:
procedure TRESTModule.RESTSchemaResources0AllowRecord(aSender: TObject;
aContext: TBaseRestContext; aDataSet: TDataset; var allowRecord: Boolean);
begin
if StrToInt(aContext.UserID) = aDataSet.FieldByName('id').AsInteger then
allowRecord:=True
else
allowRecord:=False;
end;
This will ensure that users can only see their own user record in the database, which is useful for the purposes of creating a login page, as this resource can be easily checked to determine if the user was actually authenticated or not. Remember, REST is Stateless.
If you do expose your authentication table, and have guest access, then you should do this as well on the OnResourceAllowed event to prevent any non-authenticated user from even accessing the resource:
procedure TRESTModule.RESTSchemaResources0ResourceAllowed(aSender: TObject;
aContext: TBaseRestContext; var allowResource: Boolean);
begin
if aContext.UserID = '0' then { The Guest Access DefaultUserID here. }
allowResource:=False
else
allowResource:=True;
end;
This will effectly only allow this resource to be accessed by authenticated users.
To make resources read-only for guest access, you can use this event on the OnAllowedOperations:
procedure TRESTModule.RESTSchemaResources1AllowedOperations(aSender: TObject;
aContext: TBaseRestContext; var aOperations: TRestOperations);
begin
if aContext.UserID = '0' then { DefaultUserID here. }
aOperations:=[roGet, roHead, roOptions];
end;
This will limit the guest access from doing any data modification operations on the REST Server. Simple enough.
The final event example I'll leave you with is for OnCheckParam, and can be used to set a uid field to that of the authenticated user:
procedure TRESTModule.RESTSchemaResources1CheckParams(aSender: TObject;
aContext: TBaseRestContext; aOperation: TRestOperation; Params: TParams);
begin
if aOperation = roPost then
Params.FindParam('uid').Value:=StrToInt(aContext.UserID);
end;
I haven't yet tested this with guest access, and I believe a proper check should be performed here, as I believe this is called for most requests, but I still haven't researched it enough yet.
What's Next?
In the next post, I will go through the creation of a pas2js web application client, that can use the datasets provided by this REST Server. The example will show authentication basics, and how to create a working CRUD application that works with this REST Server.