Friday, October 18, 2013

Journal Club Wasteland

Nephrology and social media has some notable successes:
  • The Kidney Group has charted a unique course with Facebook by breaking the so called rules of social media in medicine. The Kidney Group regularly highlights individual patients, and the private lives of the doctors, nurses and families of the people who work at this private practice. They have a unique voice and engage with a community.
  • Renal Fellow Network has created a website that that is remarkable for its scope, longevity and quality. They are the only nephrology website that has mastered succession, an essential element of long term success. 
  • eAJKD and NephronPower, the twin brain products of Kenar Jhaveri have demonstrated remarkable creativity in making nephrology more approachable and interesting with word games, detective stories and NephMadness.
  • Twitter has increasingly been an amazing social media destination for intelligent nephrology discussion. Two particular tweeterari standout: 
    • Ed Lerma is doing great work with his #NephPearls project. 
    • And Christos Argyropoulos (feels like I should get some kind of reward for spelling that correctly) is defining nephrology on twitter by his frequent engagement and general cleverness.
  • The Ajay Singh and The Kidney Doctor has had tremendous success, it is fascinating having an A-level scientist join the social media ghetto. His productivity and gravitas provides a unique voice. 
  • This is by no means a comprehensive list. Notable additions which immediatly come to mind include Tejas Desai, Mathew Sparks, Jordan Weinstein, Mahesh Krishnan and I am sure I have forgotten too many others. (Please post your umbrage in the comments)
Each success teaches us something about how social media works in nephrology. Failures are generally less interesting, but since we still do autopsies let's take a look at one of the highest profile failures in NephSocMedia: The CJASN journal club.

This project had a simple, straight-forward goal. Take an article from CJASN every month or so. Highlight it by giving both it, and the editorial away. Make the author available for some questions. Create an interesting online discussion of the article. Create, a national or even international nephrology journal club.

Here is a list of the discussions for this month's article on Mediterranean Diets (October):

Here are the discussions for the initiation of RRT in CKD (September):

The April '13 article on marajuana has no comments! If you can't get the internet to comment on weed, you have a problem (the likely issue here was posting article on 4/22. Total missed opportunity, 4/20 was just sitting there waiting for a post on dope).

March and February, no comments.

You get the idea. Here is the data in aggregate

The January '13 post has 3 comments, unfortunately they have nothing to do with The effects of hemodiafiltration on quality of life:

This is a failed project. Why did it fail?

I can't get past the lock on the home screen of the project

What a way to turn people away. The site literally has a padlock on the front door. The idea of an online journal club is that the wisdom of the crowd can shine through. With enough eyes and mouths all aspects of an article can be turned over and vetted. By establishing this barrier to entry they limitted participation and turned people away. Most internet forums allow everyone to view discussions and only require logging in to comment. I can't think of any reason to prevent people from browsing the discussion without first logging in.

The discussions are labelled not by first name but by e-mail address. So you know is going to be reluctant to leave a comment due to profound e-mail address embarrassment. Though I have to say I got a bit of a thrill seeing the e-mail addresses of George Bakris, John Asplin, Dave Goldfarb and other rock stars of nephrology revealed in their forum posts. In my opinion people want some degree of anonymity. CJASN did not allow that. But compared to other mistakes, I think this one is unimportant. Fixing this will not fix the Journal Club.

The funny thing is that the head of the site gets it. David Goldfarb has this blurb under latest news (note: if the link is from May 2012, time to change it from latest news):
This link is to an interesting editorial from the NYT about the value of free access and opening yourself up to critique by the public. I wonder if that blurb was Goldfarb's way of nudging the gatekeepers that put the lock on the door?

But the reason CJASN Journal Club failed is this is a project that depends on a community it does not have. I spoke with Dr. Goldfarb about this and he realized this and told me that he hoped to create the community with the journal club itself.

The idea of an online journal club is a good one and has been executed in multiple venues successfully. The one I am most familiar with is the Twitter Journal Club run by Natalie Silvey. Transcripts from the journal clubs can be found here and here. I spoke with Natalie about her experience over twitter. Some interesting tidbits:
  • choose articles with general appeal
  • you don't need to be a twitter superstar, she had only 500 followers when she launched it
  • publicity helps: the project was featured in Nature(!) and a number of her friends and followers were medics who helped stoke interest. This has started an avalanche of publicity and imitators. (see here, here and here)
  • The twitter format is very accessible to everyone
  • Since the host had no specific authority (she was a junior attending when she started it) it radiated approachability by other people. Having an author or similar authority figure can stifle discussion with presumed expertise.
One thing that I like about the Twitter Journal Club is that it's an event. It convenes at a set time and you are either there or you miss it. You can go back afterwards to scan the transcripts but the event is done. Having people discuss things in real time makes it feel more collegial and interactive, a static forum is one derivative further away from the tradition journal club. An important part of the success of the twitter journal club is a blog that holds the transcript and the opening statement. A blog can be indexed by google so it is findable and preserved. Putting the journal club behind a password blocks this utility.

In fact, the password protection gives the impression that this knowledge is off limits, It is a rejection of one of the core principles of Medicine 2.0: openness, transparency and accepting the truth that the monopoly of information that physicians used to have no longer exists.

So where does this leave CJASN Journal Club? Dr. Goldfarb admits that they are at a cross roads and shuttering the whole site is being openly discussed. I think that unless they are willing to try a whole new strategy they should just put the thing out of its misery.

If they want to try CJASN Journal Club 2.0 I would suggest:

  • No paywall, password, or mandatory e-mail collection
  • Run the journal club on twitter or in a Google Hangout
  • Make it an event by partnering with other nephrology social media hubs (essentially admit that you don't have a community so you will borrow RFN's or PBFluid's or NOD's for a session
  • Dump the forums and
  • Replace them with a blog with three or four posts per month:
    • first post: introduction to the article
    • second post: editorial or author interview
    • third post: post for the twitter event (this could be combined with #1), include the questions that will be addressed during the chat
    • Fourth post: summary after the event with the transcript and analytics from Symplur
  • Allow comments on the blog to be the replacement for the forums
I allowed Dr. Goldfarb to review this before it was posted and I gave him an opportunity to respond.
I appreciate the opportunity to read your post and reply. I have to admit that I was hoping for more participation by the nephrology community in eJC; I too envisioned something more active. We at CJASN are very receptive to this feedback, as, in truth, we have had little feedback from ASN members before your post. So we are glad that you wrote and glad to consider ways to make eJC better.

There are other measures of participation, other than the amount of discussion, that perhaps are evidence of a higher level of success than you suggest. Maybe that participation is more passive. I’m not sure what measure of “success” you are applying to some of the online activities you cite. The Renal Fellow Network is terrific but the very thoughtful posts there usually have 0 to few comments made.

Perhaps you will judge eJC as less of a wasteland if I give you some recent stats.  There are 969 people subscribed who get the monthly email. In October 44 new subscribers have enrolled so we have not stopped. For September-October we have had 350 visits by 295 unique visitors, with close to 4 page views per visit. All of those sound pretty good to me.

I also have some anecdotal tales to share. My fellow’s father is a nephrologist in India: his group discusses the eJC article every month. “He loves it”, she tells me. There were a couple of formal discussions of eJC papers at King Faisal Hospital in Riyadh under the leadership of a former NYU fellow. MAYBE there are nephrology fellows choosing our article to discuss for their fellowship program’s journal club…maybe fellows are less likely now to ask their faculty to suggest an article to present.

I have thought several times that we need to do a survey of the subscribed members and figure out what activity out there is actually ongoing, because clearly I do not have the data about those sorts of activities.

One nice service that eJC, CJASN, and ASN provide is that the article is distributed free when it would otherwise remain available only to ASN members. So if it is simply a good will gesture of ASN and CJASN to the world, I don’t think that’s a bad thing.

As for “building a community”, I was inspired by Nephrol, Kim Solez’ listserv that started in the late ‘90s and remains intact and useful. I have seen it nearly every day for more than 15 years. People have been reading it and participating in case discussions all that time…it’s self-sustaining. Maybe I have to put more time into it myself, keep the conversation juiced and pertinent and engage the authors in some dialogue, but I didn't want it to be MY opportunity to chat and exchange talk with the authors, I wanted to offer that chance to the nephrologists reading out there.

As for posting anonymously as you suggest; Nephrol is not anonymous, the participants are adults and posting serious stuff, not fooling arounds. It’s like nephrologists you never heard of getting up at a microphone at the annual meeting and saying “I’m Jones, from New York, and I’d like to know what you do with calcium phosphate stones in the salivary duct”, I didn’t think it was too important to offer anonymity. There have also been discussions showing that anonymity is responsible for many of the internet’s abuses and uglier sides. But we are willing to try that.

The journal clubs that happen at a particular time on Twitter are great and novel. Everyone has a time that they can participate and a time to browse what’s online. The one from BJU on stones and soda was useful and entertaining. But how to judge the success of those events is not clear: are you satisfied if there are 5 participants? 10? As a former chief resident, I gave up counting housestaff attendance at conferences: if one person yearning for knowledge shows up and thinks about the subject, I’ll be satisfied.

On a more minor note, I see your point about the lock on the entrance being slightly off-putting…maybe an open door would be a more welcoming symbol. ASN reasonably wants everyone getting to the site to have an ASN password. It’s free and does not require ASN membership. Perhaps some shy people see the lock and feel turned away.

We’ll have further discussions about this interesting subject. I’m grateful for this opportunity to think about what we have done and what we will do. Believe me, I am much happier being a “high profile failure” than a “low profile failure”, though I’m contesting that term, and I’m looking forward to you telling me how many people read your post and my reply.

I admire the hard work you put into your website and lectures, and your intelligent critique of the nephrologic world.


David S. Goldfarb, MD, FASN
Associate Editor and eJournal Club Editor, CJASN
Chief, Nephrology Section,
New York Harbor VA Healthcare System
Professor of Medicine & Physiology, 
NYU School of Medicine
Related Posts Plugin for WordPress, Blogger...