Best Practices for logging in MarkLogic

  • 13 May, 2024
  • By Dave Cassel
  • No Comments

An important element in software development is understanding what’s happening as your application runs. The user interface should provide responses to user actions, including informative messages when something goes wrong. For the development team to get deeper insights, we turn to logging for more detailed information. In this post, we’ll look at some options for […]

Read More


What is a reverse query?

  • 6 May, 2024
  • By Dave Cassel
  • No Comments

How do you know when your database has something new that would interest your users? How do you categorize incoming data? How do you know whose workflow to assign a new document to? With Progress MarkLogic, you use the reverse query feature. Let’s take a look. In a normal (forward) query, we define a set […]

Read More


Is MarkLogic the right database for me?

  • 27 April, 2024
  • By Dave Cassel
  • No Comments

Progress MarkLogic is an enterprise, multi-model, NoSQL database, search engine, and application server. It’s able to cover a lot of use cases, but there’s only one that matters — the one you’re trying to implement. How do you know whether MarkLogic is the right fit? If you’re considering MarkLogic, see the special offer at the […]

Read More


String interpolation in Apache NiFi

  • 22 April, 2024
  • By Dave Cassel
  • No Comments

In one of my recent posts, I talked about ExecuteScriptMarkLogic, a handy processor for getting Apache NiFi to talk to Progress MarkLogic. I’d like to share a couple little “gotchas” that we’ve run into before. I’m using ExecuteScriptMarkLogic to illustrate the point, but it applies to any processor where we use JavaScript code in the […]

Read More


Apache NiFi and Progress MarkLogic

  • 10 April, 2024
  • By Dave Cassel
  • No Comments

For years, I’ve used Apache NiFi as a data orchestration tool. Based on NiFi’s built-in scheduler, we pull data from upstream sources, send it to Progress MarkLogic, and trigger MarkLogic to take certain actions on that data. We also use NiFi to ask MarkLogic for information that is ready to process and take action based […]

Read More


Updating document quality

  • 28 June, 2023
  • By Dave Cassel
  • No Comments

A little-used Progress MarkLogic feature (from what I’ve seen) is the ability to change a document’s quality. Lowering this value will make it drop in search results, while increasing it will make it more prominent. For one client, we’re allowing subject matter experts to lower the quality of documents that aren’t very helpful. They get […]

Read More


Testing Custom Progress MarkLogic APIs

  • 28 March, 2023
  • By Dave Cassel
  • No Comments

We can use the marklogic-unit-test framework to test custom APIs hosted in Progress MarkLogic. Doing so is more of an integration test than a unit test, allowing us to ensure that HTTP inputs are correctly mapped to function parameters and that the API call works all the way through. So why don’t we only do this? A […]

Read More


Nulls and the Empty Sequence

  • 28 November, 2022
  • By Dave Cassel
  • No Comments

We recently came across a neat little gotcha that I thought was worth sharing. I’ve written before about how JSON Nodes and JS objects look the same but act differently in Progress MarkLogic; this is similar but I’m looking at null and the empty Sequence. Let’s create a really simple document to play with: ‘use strict’; declareUpdate(); […]

Read More


Apply temporal to an existing document

  • 28 October, 2022
  • By Dave Cassel
  • No Comments

MarkLogic’s temporal feature allows an out-of-the-box way to preserve copies of a document when it gets updated. You can read much more in the Temporal Developer’s Guide, but I had a need to look at a particular question recently — how do I make a non-temporal document temporal? First, let’s think about what makes a document […]

Read More


What does it mean to be a MarkLogic DBA?

  • 28 August, 2022
  • By Dave Cassel
  • No Comments

The responsibilities of a DBA are different for MarkLogic than for a traditional relational database. While the line of responsibility among a DBA, the development team, and system administrators will be drawn differently at every organization, here are some guidelines you can use. Included The activities in this section will generally be handled by a […]

Read More


MarkLogic index data types

  • 28 July, 2022
  • By Dave Cassel
  • No Comments

MarkLogic offers several types of indexes: Universal, range, triples. These indexes provide fast access to your content and can be configured to work with specific data types. MarkLogic will even do some type conversions for you. Universal Index Let’s insert a couple documents. Note the difference between the updated properties (“T” versus no “T”) and the types […]

Read More


Scoping queries in the Optic API

  • 28 July, 2022
  • By Dave Cassel
  • No Comments

Every now and then I write an Optic query that has parts that look redundant. In the example below, assume the 4V.Sample view has a “keyCode” field and that the JSON docs the view is built from have a “keyProp” property. My goal is to gather some information from a view for a specific set of values. […]

Read More


Binding multiple values

  • 28 July, 2022
  • By Dave Cassel
  • No Comments

With MarkLogic’s SPARQL queries, we can bind a value to constrain the query. Using this capability, we can gather information about something of interest. But what if I want to query against multiple values? Let’s start with some sample data. ‘use strict’; declareUpdate(); const sem = require(“/MarkLogic/semantics”) const myPred = sem.iri(“myPredicate”); sem.rdfInsert([1, 2, 3, 4, […]

Read More


SPARQL update with multiple targets

  • 28 July, 2022
  • By Dave Cassel
  • No Comments

In my last post, I talked about using the bindings parameter of MarkLogic’s sem.sparql function to look for multiple values in a SPARQL query. It turns out that approach doesn’t work for SPARQL Update. I’ll use the same sample data as my previous post: ‘use strict’; declareUpdate(); const sem = require(“/MarkLogic/semantics”) const myPred = sem.iri(“myPredicate”); sem.rdfInsert([1, 2, 3, 4, […]

Read More


TDE Template – Unknown Table

  • 28 November, 2021
  • By Dave Cassel
  • No Comments

MarkLogic’s Data Hub Central offers an easy way to create entities. As a bonus, it automatically creates a TDE Template for you, making your entity accessible through SQL or Optic API queries. However, we ran into an error with this process recently. After creating an entity and processing some sample data, a member of the […]

Read More


Populating an array from numbered fields

  • 28 November, 2021
  • By Dave Cassel
  • No Comments

As we move data from other sources (often relational databases) into MarkLogic JSON, we have the opportunity to change the way data are represented. I’ve seen a pattern a few times where the incoming source repeats a field, appending a number. I’ll illustrate with a simplified example: name,pet_name_1,pet_name2 David,Merry,Pippin Alice,Spot, Bob,Fluffy,George When I see something […]

Read More


SPARQL Update and locks

  • 28 October, 2021
  • By Dave Cassel
  • No Comments

I just learned something the hard way, so I thought I’d share. The tl;dr is that sem.sparqlUpdate runs in a separate transaction by default, which means you need to be careful about document locks. (If your response is “well, duh”, then you may not need the rest of this post. If you’ve ever had a sem.sparqlUpdate request time out when it […]

Read More


Making updates safe for parallel processing

  • 28 May, 2021
  • By Dave Cassel
  • No Comments

I’ve seen a couple cases recently where we were making modifications to a document in MarkLogic and the process was restricted to a single thread due to concerns about overwriting data. Let’s look at an example. function makeUpdates(someArgs) { let uri = ‘/some/content.json’; let docObj = cts.doc(uri).toObject(); docObj.meta.foo = updateFoo(docObj.meta.foo, someArgs); xdmp.documentInsert( uri, docObj, { […]

Read More


Comparing xs.dateTime values

  • 20 February, 2021
  • By Dave Cassel
  • No Comments

5 o’clock is 5 o’clock, right? Well, not once we start thinking about time zones. 17:00:00-05:00 (5pm here on the east coast of the US) is 22:00:00Z in London. But that’s simple enough; this is why we have the time zone expressed as part of date times. For a recent project, I needed to check […]

Read More


Processing upstream deletes

  • 28 January, 2021
  • By Dave Cassel
  • No Comments

We can load data into a data hub from a variety of upstream data sources. What if we want to delete documents that have been deleted upstream? In the ideal case we’re using a Change Data Capture system and we get notifications about those deletions, but sometimes we don’t have that. How can we detect […]

Read More


Working with JSON in XQuery

  • 6 October, 2020
  • By Dave Cassel
  • No Comments

MarkLogic supports XQuery and JavaScript as native languages. XQuery is a very natural way to work with XML, just as JavaScript is a very natural way to work with JSON. However, sometimes it’s useful to cross over and work with JSON using XQuery or vice versa. This post has some tips on using XQuery to […]

Read More


Searching for one result

  • 28 September, 2020
  • By Dave Cassel
  • No Comments

MarkLogic is both a database and a search engine. Sometimes, you know you only want one result from your search. What’s a good way to do that? Using MarkLogic’s Server-side JavaScript, here’s one way to do it: const query = …; // foreshadowing — there’s a better way! cts.search(query).toArray().slice(0, 1) We get the results of […]

Read More


MarkLogic Universal and Range Indexes

  • 28 February, 2020
  • By Dave Cassel
  • No Comments

In NiFi, FlowFiles are pieces of data that a processor needs to work on. In this case, NiFi is calling a MarkLogic query for each FlowFile. The metric shown in the graph is FlowFiles processed by NiFi over 5 minute intervals. There are peaks around 16,000, but mostly lower, which wasn’t the throughput I needed. […]

Read More


cta-bg

Looking Forward to Building a Partnership!

Let's discuss how we can help your organization