Phantom Surge in Supply from Data Feed Issue

Posted by urbandigs

Thu Jun 12th, 2014 07:21 AM

A: We are approaching the end of Q2 and I am getting many emails and calls about the recent surge in supply on the UrbanDigs Manhattan data system. I wrote about what happened a week ago when our data feed went down for 9 days at the end of May, but let me explain the phantom surge again and show you where I think we are in terms of Manhattan supply trends.

Here is a chart showing Manhattan Supply over the past 1YR:

phantom_supply_surge.jpg

The supply surge is clear in the past two weeks. The reason is because when our data feed got back online we received bulk file updates that contained many duplicate and old new listings with errors on the 'list date' field. That caused our system to add these as new supply when in fact they are not.

The UD system was engineered to "self-correct" for these types of data issues that tend to happen from time to time. The result is confusing as it is happening, like now, as it leads us to believe there was a huge spike in new supply. But over the course of 2-4 weeks the UD system will gradually remove all poisonous listings from our inventory charts -- we are starting to see this now as the big spike is starting to reverse course and come down.

To understand how our system works think of the cycle of a listing in Manhattan as a flow: from new listing --> to off mkt --> to back on mkt --> to in contract (pending) --> to sold/closed --> repeat cycle.

At all times, supply is a function of the pool of listings in Manhattan that are actively updated by the exclusive broker and the following:

-- PLUS, all new listings to hit the market today
-- MINUS, all active listings to enter contract (pending) today
-- MINUS, all active listings that are taken off the market
-- MINUS, all active listings that become stale due to lack of broker status updates
-- MINUS, all active listings that sold/closed via acris without the broker changing status to contract signed.

Add that all up and you have a dynamic system like the UD one to measure Manhattan supply. When our Rebny Listing Service feed (RLS) goes down, that cycle and flow is disrupted. When the feed comes back, issues like this can occur.

As I wrote about last week the answer is to WAIT AND LET THE UD SYSTEM SELF CORRECT TO THE RIGHT LEVEL -- IT IS STARTING TO WEED OUT THE BAD STUFF NOW.

But it still needs time so we need to wait a few more weeks and then we will go back and try to remove the bad stuff manually and smooth out the chart.

I hope this helps to explain what is going on, why our system is doing what it is, and what we plan to do to resolve this. My apologies for the inconvenience and temporary distortion in Manhattan supply trends.


CAPTCHA Image