Introduction
In most of the reviews for this post, I was asked why choose a graph database over something else? This is a hard question to answer since my experience right now is limited on the graph database side. My guess is you’re wondering the same thing, so this is my best answer to date.
At this point in my career, I would only choose a relational database if I was writing financial software. Relational databases are very rigid so you need to really make sure you understand your data upfront. Changing the database is a big effort, but the database when designed correctly will give you very high levels of integrity and you can get good performance. These databases are hard to distribute and scale so you tend to end up with single instances that are very large and require replication for backup. The cloud providers have relational databases today that are supposed to scale, but I have no experience with them.
I spent 7 years building products on MongoDB (document database). These databases come with the promise of distributed scalability and lots of data flexibility. The distributed scalability is very complicated (since it requires sharding) and they have never been as reliable as the relational databases. If data integrity is not an issue and your data is constantly changing, these databases are great.
I am now learning Dgraph (graph database) and GraphQL. I am really liking what I see. Dgraph is a distributed scalable database that works in the cloud and built from the ground up for that environment. It does require a schema so you can get the data integrity, but it’s not as rigid so you have more flexibility to make changes. Dgraph supports transactions, but I have not played with that yet.
Up & Running With Dgraph
What We Will Learn
This post is intended to help you get up and running quickly with Dgraph and GraphQL. I wish this post existed when I started several months ago. There is a world of documentation out there for GraphQL, but none of it made sense to me until I understood the material I will present in this post.
In this post, I will teach you the basics of defining a GraphQL schema, and then show you the GraphQL CRUD API that Dgraph generates from that schema. After that, I will show you how to access the API using a nice graphical, interactive, in-browser GraphQL IDE called GraphQL Playground. At the end of this post, I listed questions reviewers had and answers from the Dgraph engineering team.
GraphQL Schema
The inspiration for this post is a project I’ve been working on with Dgraph called Travel. The Travel project is a starter kit similar to the Service project I use to teach application development in Go. The Travel project is focused on learning how to write production applications in Go using Dgraph and GraphQL.
The Travel project provides information for a city where four points of data are gathered and related.
Listing 1
type City {
id: ID! # mandatory, scalar type, lookup
lat: Float! # mandatory, scalar type
lng: Float! # mandatory, scalar type
name: String! @search(by: [exact]) # mandatory, scalar type, searchable
advisory: Advisory # not-mandatory, custom type
weather: Weather # not-mandatory, custom type
places: [Place] # not-mandatory, custom type, list
}
type Advisory {
id: ID! # mandatory, scalar type, lookup
continent: String! # mandatory, scalar type
country: String! # mandatory, scalar type
country_code: String! # mandatory, scalar type
last_updated: String # not-mandatory, scalar type
message: String # not-mandatory, scalar type
score: Float! # mandatory, scalar type
source: String # not-mandatory, scalar type
}
type Place {
id: ID! # mandatory, scalar type, lookup
address: String # not-mandatory, scalar type
avg_user_rating: Float # not-mandatory, scalar type
city_name: String! # mandatory, scalar type
gmaps_url: String # not-mandatory, scalar type
lat: Float! # mandatory, scalar type
lng: Float! # mandatory, scalar type
location_type: [String] # not-mandatory, scalar type, list
name: String! @search(by: [exact]) # mandatory, scalar type, searchable
no_user_rating: Int # not-mandatory, scalar type
place_id: String! # mandatory, scalar type
photo_id: String # not-mandatory, scalar type
}
type Weather {
id: ID! # mandatory, scalar type, lookup
city_name: String! # mandatory, scalar type
description: String # not-mandatory, scalar type
feels_like: Float # not-mandatory, scalar type
humidity: Int # not-mandatory, scalar type
pressure: Int # not-mandatory, scalar type
sunrise: Int # not-mandatory, scalar type
sunset: Int # not-mandatory, scalar type
temp: Float # not-mandatory, scalar type
temp_min: Float # not-mandatory, scalar type
temp_max: Float # not-mandatory, scalar type
visibility: String # not-mandatory, scalar type
wind_direction: Int # not-mandatory, scalar type
wind_speed: Float # not-mandatory, scalar type
}
Listing 1 shows the complete GraphQL schema for the application. I have added comments that break down each field (predicate) in the schema definition. That should give you a good starting point for understanding the syntax you see. I will break down each of the unique aspects of the syntax a bit more.
GraphQL has a set of default scalar types.
Listing 2
Int: Signed 32‐bit integer.
Float: Signed double-precision floating-point value.
String: UTF‐8 character sequence.
DateTime: string in RFC3339 format
Boolean: true or false.
ID: Auto-generated unique id.
Listing 2 shows the set of default scalar types provided by GraphQL. These are the basic types provided by any database or programming language.
Listing 3
type City {
id: ID! # mandatory, scalar type, lookup
}
Listing 3 shows the declaration of the id predicate using the scalar type ID. This scalar type auto-generates a unique ID for any node of data that is added to the database. Dgraph physically organizes the data based on this ID so a lookup can be performed quickly. This type is also a marker that Dgraph uses to build a full GraphQL CRUD API for the type.
Listing 4
type City {
id: ID! # mandatory, scalar type, lookup
lat: Float! # mandatory, scalar type
}
type Advisory {
continent: String! # mandatory, scalar type
}
Listing 4 shows three predicates in the schema using the exclamation point (!). This indicates to the database that data for these predicates is required for any add or update operation. When the exclamation point is missing, the predicate is allowed to be null.
Listing 5
type City {
places: [Place] # not-mandatory, custom type, list
}
type Place {
location_type: [String] # not-mandatory, scalar type, list
}
Listing 5 shows two predicates declared as a list by using square brackets around the type declaration.
Listing 6
type City {
name: String! @search(by: [exact]) # mandatory, scalar type, searchable
}
Listing 6 shows a field declared as searchable by an exact string search. There are other options to choose from.
Listing 7
input StringExactFilter {
eq: String # Equal to
le: String # Less than or equal
lt: String # Less than
ge: String # Greater than or equal
gt: String # Greater than
}
input StringTermFilter {
allofterms: String # Match all words provided: "sydney melbourne"
anyofterms: String # Match any words provided: "sydney melbourne"
}
input StringFullTextFilter {
alloftext: String # Match all words provided: "sydney melbourne"
anyoftext: String # Match any words provided: "sydney melbourne"
}
input StringRegExpFilter {
regexp: String # Match any city with ney: "/.*ney.*/"
}
Listing 7 shows the different string-type searches you can choose from based on input types defined by Dgraph. When a field is searchable, Dgraph builds GraphQL query API’s for the type based on the search filter selected.
Listing 8
type Example {
temp: Float @search
pressure: Int @search
sold: Boolean @search
}
Listing 8 shows you how to define searchable predicates for some of the other scalar types.
Listing 9
input FloatFilter {
eq: Float
le: Float
lt: Float
ge: Float
gt: Float
}
input IntFilter {
eq: Int
le: Int
lt: Int
ge: Int
gt: Int
}
Boolean doesn’t have a special input filter type.
Listing 9 shows the other scalar type searches you can choose from based on input types that are defined by Dgraph. The Boolean type does not have a special input type defined.
Running Dgraph
The best way to run Dgraph on your local machine is to use Docker. If you don’t have Docker installed yet, then please follow this guide from Docker.
As of the writing of this post, version 20.03.1 is the latest release of Dgraph. To download this version of Dgraph, just use the docker pull command.
Listing 10
$ docker pull dgraph/standalone:v20.03.1
OUTPUT
v20.03.1: Pulling from dgraph/standalone
5bed26d33875: Pull complete
f11b29a9c730: Pull complete
930bda195c84: Pull complete
78bf9a5ad49e: Pull complete
56d14a4dc4d7: Pull complete
b2e6192c62a3: Pull complete
02dca55cd04c: Pull complete
ca9eba547da7: Pull complete
e4c262d6c0b0: Pull complete
Digest: sha256:22d2b55b3cb4d23032de2aa8b484ca83bebcfbba102010c259b332a239b676a6
Status: Downloaded newer image for dgraph/standalone:v20.03.1
docker.io/dgraph/standalone:v20.03.1
Listing 10 shows the docker pull command that downloads a standalone version of Dgraph and prepares Dgraph to run inside a container. Once all the different file layers are downloaded, run the docker images command to validate the pull.
Listing 11
$ docker images
OUTPUT
REPOSITORY TAG IMAGE ID CREATED SIZE
dgraph/standalone v20.03.1 8ed9567a0f71 11 days ago 148MB
Listing 11 shows the call to docker images and the expected output for the Dgraph image. To validate this image works, spin up a container.
Listing 12
$ docker run -it -p 8080:8080 dgraph/standalone:v20.03.1
OUTPUT
Warning: This standalone version is meant for quickstart purposes only.
It is NOT RECOMMENDED for production environments.
2020/04/10 14:07:40 Listening on :8000...
[Decoder]: Using assembly version of decoder
Listing 12 shows the docker run command to use to get Dgraph up and running. Notice the message saying this version of Dgraph is not meant for production use. Use this version for local development and testing. Port 8080 is the port exposed for GraphQL.
Create Schema
With Dgraph up and running, the next step is to create a schema. The quickest way to create the schema is to run a curl command.
Listing 13
curl -H "Content-Type: application/json" http://localhost:8080/admin/schema -XPOST -d $'
type City {
id: ID!
advisory: Advisory
lat: Float!
lng: Float!
name: String! @search(by: [exact])
places: [Place]
weather: Weather
}
type Advisory {
id: ID!
continent: String!
country: String!
country_code: String!
last_updated: String
message: String
score: Float!
source: String
}
type Place {
id: ID!
address: String
avg_user_rating: Float
city_name: String!
gmaps_url: String
lat: Float!
lng: Float!
location_type: [String]
name: String! @search(by: [exact])
no_user_rating: Int
place_id: String!
photo_id: String
}
type Weather {
id: ID!
city_name: String!
description: String
feels_like: Float
humidity: Int
pressure: Int
sunrise: Int
sunset: Int
temp: Float
temp_min: Float
temp_max: Float
visibility: String
wind_direction: Int
wind_speed: Float
}'
OUTPUT
{"data":{"code":"Success","message":"Done"}}
Listing 13 shows the curl command that creates the schema. Copy, paste, and run that curl command in your terminal. After it completes, you should see the output as presented.
GraphQL Playground
The GraphQL Playground is a graphical, interactive, in-browser GraphQL IDE, created by Prisma and based on GraphiQL. This is a nice tool for verifying that the schema and data are properly managed in the database. The link provides access to desktop apps for the three major operating systems.
Once you download the application and get it running, you should see this.
Figure 1
Select the URL ENDPOINT option and type http://localhost:8080 in the text box as seen in figure 1. Then hit the OPEN button.
Figure 2
Assuming you left Dgraph running at listing 12, you should see no connection errors, an empty workspace, and two tabs on the right as shown in figure 2.
Since the “dark” theme is very dark, I am going to change the theme to “light”. To do that, select the gear icon in the top right corner of the application and change the editor.theme setting from "dark" to "light" and hit save.
Listing 14
"editor.theme": "light",
After changing the theme, the GraphQL Playground should look like this.
Figure 3
Figure 3 shows how GraphQL Playground should look after changing the theme to "light".
The two tabs on the right, DOCS and SCHEMA provide schema information. The DOCS tab is an interactive, hyper-linked view of the CRUD API. The SCHEMA tab provides a flat view of the entire schema.
Figure 4
Figure 4 shows you a partial view of the flat schema from the SCHEMA tab. This schema not only shows the data definitions you provided, but all the types and the CRUD API generated by Drgaph.
CRUD API
There are two types that are important since they represent the CRUD API.
Listing 15
type Mutation {
addCity(input: [AddCityInput!]!): AddCityPayload
updateCity(input: UpdateCityInput!): UpdateCityPayload
deleteCity(filter: CityFilter!): DeleteCityPayload
addAdvisory(input: [AddAdvisoryInput!]!): AddAdvisoryPayload
updateAdvisory(input: UpdateAdvisoryInput!): UpdateAdvisoryPayload
deleteAdvisory(filter: AdvisoryFilter!): DeleteAdvisoryPayload
addPlace(input: [AddPlaceInput!]!): AddPlacePayload
updatePlace(input: UpdatePlaceInput!): UpdatePlacePayload
deletePlace(filter: PlaceFilter!): DeletePlacePayload
addWeather(input: [AddWeatherInput!]!): AddWeatherPayload
updateWeather(input: UpdateWeatherInput!): UpdateWeatherPayload
deleteWeather(filter: WeatherFilter!): DeleteWeatherPayload
}
Listing 15 shows the Mutation type that contains the mutation API that was generated based on the schema. You’ll notice there is an add, update, and delete function for each type. This is because the ID predicate was added to each type in the schema.
Listing 16
type Query {
getCity(id: ID!): City
queryCity(filter: CityFilter order: CityOrder first: Int offset: Int): [City]
getAdvisory(id: ID!): Advisory
queryAdvisory(filter: AdvisoryFilter order: AdvisoryOrder first: Int offset: Int): [Advisory]
getPlace(id: ID!): Place
queryPlace(filter: PlaceFilter order: PlaceOrder first: Int offset: Int): [Place]
getWeather(id: ID!): Weather
queryWeather(filter: WeatherFilter order: WeatherOrder first: Int offset: Int): [Weather]
}
Listing 16 shows the Query type that contains the query API that was generated based on the schema. You can see there is a get and query function for each type. The get function exists because the ID predicate was added to each type in the schema. Each type will get a query function by default. Don’t worry about indexes, the first time a query function runs on a specific predicate, an index is created.
It’s important to note that all interaction with the database will be through the GraphQL CRUD API. If you don’t have a function that you need, you will need to change the schema to have Dgraph generate it for you or you will need to add a custom function yourself.
CRUD Operation
Now I will execute the addCity, getCity and queryCity functions from inside the GraphQL Playground to show you how to access this API in a GraphQL format.
Listing 17
mutation {
addCity(input: [{
name: "sydney"
lat: -33.865143
lng: 151.209900
}])
{
city {
id
name
lat
lng
}
}
}
Listing 17 shows how to call the addCity function to add a city to the database. Notice how the API is structured around the type system defined by Dgraph in the schema.
Listing 18
type Mutation {
addCity(input: [AddCityInput!]!): AddCityPayload
}
input AddCityInput {
advisory: AdvisoryRef
lat: Float!
lng: Float!
name: String!
places: [PlaceRef]
weather: WeatherRef
}
type AddCityPayload {
city(filter: CityFilter, order: CityOrder, first: Int, offset: Int): [City]
numUids: Int
}
input CityFilter {
id: [ID!]
name: StringExactFilter
and: CityFilter
or: CityFilter
not: CityFilter
}
Listing 18 shows all the supporting types used to define the addCity function. Having access to the schema and knowing how the APIs are structured will help you understand how to use the full API generated by Dgraph.
Copy and paste the code in listing 17 into GraphQL Playground and click the play button.
Figure 5
Figure 5 shows the addCity function call and the result in the GraphQL Playground.
To retrieve the city from the database, there is a lookup function named getCity, and a query function named queryCity.
Listing 19
query {
getCity(id: "0x2") {
id
name
lat
lng
}
}
Listing 19 shows the getCity function call. Copy and paste this call into the GraphQL Playground and click the play button.
Figure 6
Figure 6 shows the getCity function call and the result in the GraphQL Playground.
Listing 20
query {
queryCity(filter: { name: { eq: "sydney" } }) {
id
name
lat
lng
}
}
Listing 20 shows the queryCity function call. Copy and paste this call into the GraphQL Playground and hit the play button.
Figure 7
Figure 7 shows the queryCity function call and the result in the GraphQL Playground.
Shutdown Dgraph
To shutdown the database, find the container id and then use the docker stop command.
Listing 21
$ docker ps
OUTPUT
CONTAINER ID IMAGE COMMAND
fdf70d833dd8 dgraph/standalone:v20.03.1 "./run.sh"
$ docker stop fdf70d833dd8
$ docker rm fdf70d833dd8
OUTPUT
Listing 21 shows how to find the id of the running Dgraph container and then stop and remove it. You can also hit C in the terminal window running Dgraph to perform these same actions.
Conclusion
In this post, I showed you how to run Dgraph locally, the basics of defining a GraphQL schema, how to use the generated GraphQL CRUD API, and how to install and use a tool called GraphQL Playground. All these things are necessary to get up and running with Dgraph for application development.
In future posts, I will show you how to leverage Dgraph, GraphQL, and the auto-generated CRUD API to build production based applications in Go. I’ll do that with the Travel project that is available today.
For now, take the time to generate your own schema and look deeper at the schema documentation provided by Dgraph. There are interfaces, enumerations, and inverse edges that I didn’t cover in this post. There are other interesting types and details in the documentation so do take a look.
Questions
Do we need to add an ID predicate on every type we are defining?
You aren’t required to add an ID to a type. Types with IDs are provided a get query function and update mutation function because they can be identified uniquely.
Given a type in the schema named Product with an ID predicate:
Query Functions Provided:
Mutation Functions Provided:
- addProduct
- updateProduct
- deleteProduct
Given a type in the schema named Product without an ID predicate:
Query Functions Provided:
Mutation Functions Provided:
Is there a way to have a user defined ID?
You can use @id on a String! field for user-defined IDs. See the example in https://graphql.dgraph.io/docs/walk-through-example/. This ensures that 1) the field is unique and 2) automatically adds the hash index to the field for searching.
How would user defined ID’s affect performance and api generation?
User-defined IDs’s get query would take the String as input for the query. Mutations would require an internal lookup of the user-defined ID to find the node.
The GraphQL API would allow you to query and mutate the type based on your user-defined ID instead of a hexadecimal number for the ID.
Is the mutation and Query API a standard GraphQL API setup or is this something special in Dgraph?
Query and Mutation are standard operations in GraphQL.
Do all databases that support GraphQL provide the CRUD api’s like Dgraph?
With Dgraph, you’ll define your GraphQL schema and Dgraph automatically generates a full GraphQL API to access the database. With other database systems, GraphQL support is typically done outside the database where you must also set up your own custom resolvers to translate GraphQL queries into the underlying database query language.