Archive for the ‘Computers and Technology’ Category

What Was I Going to Say?

Thursday, November 1st, 2018

I can’t recall because I came to my blog and there were more updates and one of them hung and then I had to disable one, and do some manual updates, and sometimes the internet here is so s. . . . l. . . o. . . w that I give up after a while. Good grief.

Anyway I was going to say whatever it was. I’m sure it would have been AMAZING. So instead, I’ll say, I hope everyone who observes Halloween had a wonderful time. Oh, and I’m going for broke on the covers for the Street Witch series. The photographer/cover person did the shoot last week with all three models, and I have been through all 750+ photos and selected my top several for each.

The next step is to take a look at the processed favorites and start talking with them about which ones to go with. I will be an awful tease and say that the images are beyond awesome. All three models nailed what I was going for and, of course, so did the photographer.

I am very excited!

The first book is close to being ready to send for editing so I’m hopeful the book will be out in early 2019.

Maybe later I’ll remember what I was going to write about.

Share

Writer’s Toolbox

Sunday, October 21st, 2018

The Search for the Perfect Writing Tools

Writers need the right tools for their work, and boy do I have opinions on this. With respect to software, my opinion is that the people developing these tools spoke to anywhere from zero to one actual writer. In fact, I think they developed whatever their programming environment made possible and then justified that result to writers. Which means the tools in general lack the flexibility required when actual people use those tools.

Until about a year ago, I wrote in WordPerfect. I run a Windows virtual machine on my desktop and laptop so I can continue to use a tool that actually IS pretty darn flexible and doesn’t impose the Microsoft “YOU WILL WRITE THIS WAY BECAUSE” paradigm. Oh, and formatting. The formatting is so awful. Anyway I have been increasingly annoyed and irritated by the VM I use. (Parallels).

Parallels mostly works but when something goes wrong or when you need to contact the company for any reason the experience is dreadful and I have been searching for a WordPerfect alternative because in my perfect world, I never have to give the Parallels company another dime of my money. If anyone from Parallels reads this; I hate your company and I hate your broken, opaque website and I hate that you hate your customers.

For years I have  been looking for a tool that would allow me to write in MacOS. Because I’m telling you Word ain’t it. I have to have MS Office in order to be compatible with the rest of the world, but I really really really hate Word. That said . . . The MS office suite tools that allow you to open and edit documents on any iOS device are wonderful. Editors, copy-editors, and proofreaders tend to work in Word, so this ability is a must have for me, and in this case, MS nailed it.

This tool works. So, for every purpose except writing fiction, thumbs up to Office 365 and its MacOS/iOS tools.

I did try Scrivener. I really did but its not a tool that works for the way I write. I envy all the people who love this tool and haven’t had to fight their natural writing process to use it. I just can’t

I tried a bunch of other tools; Open Office and Pages were total fails. I tried several apps that proclaimed they were great for writers and none of them were, and most did not mean “great for people writing fiction of 100,000 words.”

Ulysses is the only tool I’ve found where I started a writing project and was able to complete it in that tool instead of moving it to WordPerfect.  I wasn’t wasting hours trying to do simple things like work with individual chapters sometimes or with the entire document other times, oh, and being able to assemble and reorder chapters. I didn’t have to fight to have an environment that was pleasing to my eyes and it exports to a docx format (and several others) without forcing me to spend much or any time reformatting. It’s good enough to send to editors etc.

I wrote an entire novella in the tool and it was fairly seamless. I’ve written Bound in Smoke in Ulysses, and that book is just about done. I haven’t used WordPerfect in ages. I have a few things I wish it would do but nothing that keeps me from writing and getting the output I need.

Share

Customer Service – A Rant

Sunday, August 26th, 2018

If you’re reading this, congratulations to us all because it means my move to a new webhost was successful.  I’d been with my previous webhost for over ten years and they were really stellar. If there was a problem it got fixed quickly and courteously. I feel it’s not unrelated, given the story I’m about to tell you, that the person who handled most of the issues I had was a woman who recently left that company.

I was surprised when the security certificate on my site didn’t/couldn’t renew. I contacted my host about why it hadn’t renewed and the response was “You don’t need a cert because you don’t sell anything on your website.”

Notice that this does not respond in any way to my request. Note as well that the customer service rep is wrong. Deeply wrong, in fact.

For the last two years at least Google has been downranking websites that don’t run on SSL (That is, has a security certificate and is https:// rather than http://) and by now users are seeing warnings when they land on a site that isn’t running on SSL. Not to mention an expired cert will cause all kinds of problems for a website.

This is an experience women have all the time. They get worse customer service and more roadblocks in the way of that service. I opened a ticket about why an existing cert hadn’t renewed and the answer to me, someone with an obviously female name, was to question my need for a service I was using. At every step, in fact, my issues were not taken seriously.

On top of this add that these tech support people did not understand certs and how their backend requires they be set up for the customer, and the problem is even worse.

What’s in a Name? Customer Service

The short version of this tale is that I spent from the 14th to 20th explaining over and over what they needed to do and telling them that their changes actually made things worse and had ended up breaking my email. There  was a string of responses over those days that were all basically, we fixed this, please reply to the cert email we are sending you, an email  I was not getting (if they were actually sending it which I doubt because they were doing it wrong) because they broke my email. And if was wasn’t that, it was, please send us the error messages and tell us exactly what the problem is and I kept saying, OK, but look at in the thread of this ticket where I already did that, but here it is AGAIN.

Carolyn Not Charles

It was worse than that, way worse. And I’m just going to say that if my name was Charles Jewel my cert would have been renewed on the first day and I would still be their customer.


Share

Writing and Technology

Saturday, May 19th, 2018

Writing and Technology Carolyn Style

I recently bought an iPad Pro and an Apple Pencil, since some writers I know had switched entirely and my MacBook Pro is aging a bit. In addition, I have at last found a Mac app that I can write in without being incredibly frustrated and annoyed. My novella, The Viscount’s First Kiss, was written entirely using the Ulysses app.  Ulysses exports to Word, PDF, and HTML, among other formats. It syncs seamlessly between all my devices. I’m fifty thousand words into a novel with Ulysses.

I have some nitpicks, but  in the main I like it. In case you’re wondering, for essentially my entire writing career I’ve been using WordPerfect. When I switched to Mac OS, I installed a Windows virtual machine so I could continue using WordPerfect. There’s a lot about WordPerfect I miss, but Ulysses is the first acceptable replacement I’ve found that works for me.  Yes, I tried Scrivener but it just doesn’t work for me. I know it does for others and more power to you. I keep hoping that Scrivener will address its controlling propensities and stop interfering in the way I want to write. With Ulysses, I started writing and just kept going. I didn’t have to fight with the interface, and it took me about thirty seconds to read about how to make things look the way I want them to.

Anyway,  a while back, my MacBook Pro underwent some kind of glitch that required a complete reformat and restore from backup. To be honest I haven’t bothered to fix the Windows virtual machine on it since I bought the iPad Pro.

What I love About The iPad Pro For Writing

It’s a lot lighter than my MacBook Pro, but about the same dimensions other than thickness.  I bought the Apple keyboard cover, which is pretty sweet. That keyboard is about the same size as the keyboard for the MacBook Pro. When I’m writing on it, I love that it’s a touch screen, too.  I am able to rest the iPad Pro on the exercise bike do-hickey for holding devices, and have been able to dictate while I exercise.  This gives me an additional hour of writing time, which is nice.  I’ve used the Apple pencil a little less than I thought.  I find the documentation for my notebook apps to be pretty bad, and I hate spending more than a few minutes trying to figure out how to do something. However, I have used it for some note booking, for example, and that has been absolutely awesome. Writing longhand is a different experience and a different way of unlocking the writing brain.

What I Don’t Like About Writing on The iPad Pro

iOS is simply not quite as functional as Mac OS there are a couple of things I cannot do on the iPad Pro that I can do on Mac OS, predominantly having to do with one or two apps that I like. The reasons this is a con for writing, is that I want to travel with the iPad Pro and not have to bring the MacBook Pro.

Slight Segue Into Dictation

A couple of years ago I started using dictation because I wanted to make sure I preserved the functionality of my fingers and wrists and because once you settle into the dictation it can be a lot faster than  typing by hand. I had a disaster once that I resolved by dictating a massive number of edits that had been lost. If I hadn’t done that I would not have been able to finish my book on time. For my emergency, I used the built-in Apple dictation. It was not perfect but it worked and well enough I that I bought Dragon dictation both for Mac OS and Windows. Unfortunately, neither was seamless or effortless. Yuck.

What I don’t understand is why Apple insists on using straight quotes in apps where curly quotes are available and enabled. Word corrects this  annoyance, but the Ulysses app does not. Straight quotes are an abomination in fiction. Dragon, however, fixes those straight quotes, so I’m stuck using Dragon When I am writing on the iMac. Speech recognition in Dragon for Mac is pretty awful. Considering how expensive the program is, Nuance should be ashamed. In iOS, I have to use Apple dictation and for some reason that is completely beyond me, when the keyboard cover is attached to the iPad Pro you cannot dictate. WTF, Apple? When I’m on the bike, I remove the cover so I can dictate.

That said, I have watched Apple dictation perfectly transcribe my words and then change them to near gibberish as they are processed. Why? Why? Why is there no setting that allows me to tell it that I am dictating fiction and to stop changing everything to present tense or using pronouns that aren’t the pronouns I used. In addition, there are several  punctuation conventions for fiction that Apple and Dragon completely blow over. It drives me batty. For dialogue, a comma, followed by a close quote should not be followed by a capital letter unless that word actually requires a capital letter: “More coffee,” Mrs. Smith said. But: “More coffee,” she said with a desperate smile. NOT “More coffee,” She said.

It’s annoying as hell. Oh and also for some reason on my computers, both Dragon and Apple dictation randomly capitalize words. Really?

 All in All

Nevertheless, I have not used the MacBook Pro in quite some time. The combination of the iPad Pro, Ulysses app, and the Apple Pencil overlap with several functionalities: a MacBook Pro, a paper notebook, printed MSS, and pens and ink. There was a day when I forgot to bring the iPad Pro with me but I had my regular iPad. I was able to work exactly where I left off. If I had to, I could write on my iPhone.

That said, the RWA National Conference is coming up and I haven’t decided if I’m also going to bring the MacBook Pro. If I don’t, there is an app I use daily that I will have to skip. It wouldn’t be a big deal, though. So I think probably not. My board service means that I fly several times a year and that means I have learned to hate flying, checked luggage, and heavy purses.

Overall, I really like the Ulysses app and hope that they work on the needs of fiction authors. I LOVE LOVE LOVE the portability and lightness of the iPad Pro, and the ability to do longhand writing with it. This combination does most of what I need. It’s just short of perfect.

I find it ironic that so many apps, including Microsoft Word, talk about how great they are for professional writing when in fact they fall down in crucial ways that professional fiction writers need. I would pay a lot of money for WordPerfect as a native Mac App.

 

Share

Writing Stuff I’ve been Doing Lately

Friday, November 24th, 2017

I’ve been wanting to sell books directly for quite some time. I was able to sell print books from my site, but I removed the links to doing so because it was a PITA mostly because of the combination of PayPal functionality and people who just won’t look in their spam folders. . . .

The publishing environment being what it is, and given that most of the major platforms that let you sell books directly have addressed their issues with payment processing and tax collection and remittance, I really wanted to try this again.

Herewith my Tale

The important thing to understand here is that I have a low tolerance for bad instructions and almost no time to fuss with stuff. This is key to understanding why I ended up where I am. (Happily so!)

I started looking once again at all the systems that profess to allow you to sell books directly to consumers: Selz, eJunkies, Shopify, SquareSpace, and several open source systems that are free to install via my website host.

I had tried to set up Selz before and couldn’t bridge the last mile then or now. The whole interface was confusing to me, and they charge a monthly fee PLUS a percentage. And of course the cheapest plans offer so little that they are useless as options. I would have to sell a lot of books per month just to recoup the monthly fee, and I don’t even know if I will sell any. Selz was a no.

eJunkies doesn’t have a monthly fee, and they have a processing fee, which I was okay with, but then, its interface was confusing and the documentation was terrible and not specific enough. But I wanted to use BookFunnel to deliver my files and then I just got annoyed that it, too, isn’t really set up for what I wanted to do. I mean, you sell a book, and you need to deliver the buyer the format they want. Not ALL the formats. Anyway, I couldn’t confirm that I could do what I wanted so I moved on.

Shopify also did not have clear information about what they offer and JFC it’s expensive so NO.

I tried SquareSpace but, I’m sorry, the documentation is horrible and unclear, and I couldn’t even begin to figure out how to make the theme for digital products do anything appropriate for books. Abandoned.

I looked at all the shopping carts available via my website cPanel and decided to try Abante because it sounded like it would work. The instructions/documentation were written by someone who does not speak English as a first language well enough to provide clear documentation. I have some facility with interpreting the sentence structure of the country of origin, but this was frustratingly unclear. Abante even comes with a category for books so you’d think the product would be set up to actually sell digital books. I did the install, and spent way too much time setting everything up only to be unable to make the product allow me to let users select which digital version they want to have. They also don’t integrate with BookFunnel. Fast forward through a lot of frustrating googling and I realized that this product in fact cannot do this, and they apparently have no interest in addressing the issue.

Frustration is Frustrating

Yeah, I was getting pretty damn frustrated. I mean, LOTS of people want purchasers to be able to select which of a variety of digital files they actually want. People who sell photographs, for example.

So, then I looked at WooCommerce, and lo-and-behold, their documentation was clear enough, though not completely clear, and googling yielded a lot of great videos and other step-by-step explanations of how to do exactly what I wanted to do. WooCommerce is free, but you pay for certain additional functionality. I’ll skip ahead a bit and say that you could sell books without ponying up for anything, but there are a couple of useful extensions. And it happens, they were all on sale when I was getting this set up.

I installed WooCommerce on the domain I wanted to use, since it was already a WordPress site, and literally, within 15 minutes I was setting up books with the ability for customers to select ePub, Mobi, or print. BookFunnel integrates with them. When I ran into a question, the WooCommerce documentation had the answer about 60% of the time and the rest of the time google landed me on an excellent video step-by-step.

Trying to Cross the Last Mile

There were three stumbling blocks: The first is one that WordPress and WooCommerce need to address because it’s completely stupid; which is the ability to easily allow WordPress to upload additional file types. I get why they wouldn’t with regular WordPress, but WooCommerce is set up to sell digital files. The product needs to easily enable additional file types.

This is NOT easy and not documented. However, I found two plugins that offered the ability to add file extensions. Add WP Mime Types is what worked, but at first I thought it didn’t. I had to Google what exactly to enter to enable ePub and Mobi,but HERE IT IS so you don’t have to suffer:

epub = application/epub+zip
mobi = application/x-mobipocket-ebook

Just enter those two lines in the WP Mime Types plugin, and boom. Done, with one gotcha. I was expecting that when I went to select files in the WordPress media uploader, files with the extension ePub or mobi would no longer be grayed out. They remained grayed out and I was about to start cursing. Then I dragged the files to the window and then after I was done dealing with my intense frustration I looked up, and the files were there, listed in the media gallery.

WOO-Fricking-Hoo!!!

After that, it was easy to connect the ePub file to the ePub option and the mobi file to the mobi option, and set up the print option to not be a digital download, and I was essentially over the last blocker.

The next issue was how to make WooCommerce connect Book series, and that did require an extension to do this seamlessly, but in hindsight, I think it could be done, less elegantly, without the extension. I did not realize (and there was nothing in the documentation about this) that Categories of products do not actually connect to a product in the database until the product is published. I wish that weren’t the case, because you can’t easily sandbox efforts to see if you’ve made a sensible series of categories and sub-categories.

They have a sub-theme for books, which I went ahead and bought. But that is also not entirely necessary. It also took me a while to realize that you have to create a thumbnail image for each category. If you don’t, you end up with big ugly blank image markers on the shop home page. The intention is that users see an image that represents, say, All Books and others for Historical Romance, Paranormal Romance, each series, etc. Which they click on to see the various categories and subcategories, with, for me, series names being a subcategory of the series genre.

I am not a graphics person, but I managed to kludge together some images that solve the issue OK-enough. But this is something I’ll have to address as an improvement.

Payments

The last issue was payment. It was easy to set up Stripe, and I already have PayPal Standard. WooCommerce made this really really simple. No real surprise, Apple Pay isn’t working. Stripe integrates with Apple but the final hurdle is over at Apple and their instructions are unclear. So I’ll disable that for now, and work that out in the near future.

So, technically, I had everything set up in a day, and I THINK I’m ready. But a bit nervous about pulling the trigger and re-enabling my website’s shop menu tab and sending everyone to my WooCommerce set up. I need to fuss with a few more things.

Updated to add: I did figure out the ApplePay issue. I had to ftp a specific file to a specific directory and then wait for Apple to agree the file was there. So, ApplePay appears to be working.

Share

Mind The Gap

Friday, November 24th, 2017

There is a happy ending to my story so I’m able to tell you about the nightmare of the past 12 hours. Being the geek that I am, I have an iPhone X. In fact, it arrived just this Tuesday, so on Thursday, my phone (which I LOVE so far) still had that new phone smell. Folks, the iPhone X is a religious experience for me. I really, really, really like this phone.

Thursday evening I was getting in the car for a short but necessary drive, and I put my phone down in the center console near the middle of the seat. Only the phone slipped into the gap between the seat and the side of the console and then slid completely out of sight. Completely. It was NOT underneath the seat. After 45 minutes of me, my son, and my sister trying to even see where it was and get it out, we failed. I resigned myself to having to go to my mechanic to have the front seat removed in order to retrieve the phone from the innards of the driver’s seat.

Important note: my son insisted that the phone was underneath the center console. To which I replied, “No, it can’t be. I saw it slide underneath the seat.” In the opposite direction.

My phone and I were forced to spend the night apart for the first time in our nascent relationship. The horror. Also, the horror of realizing that I would not be able to easily access any of the sites where I have enabled 2-Factor Authentication and ACK HOW DOES ANYONE FUNCTION WITHOUT A PHONE?????

This morning, I had to call ON A LAND LINE— what even is that? —to tell my mechanic that I had a car emergency and to be prepared for my immanent arrival. I then drove immediately there. Two people tried to find my phone. We were about to set about removing the front seat when a third person said he’d try, and after calling my phone several times he too said the phone was under the center console. He fetched a thingamajig and two minutes later he handed me my phone. They didn’t even charge me.

I HAVE MY LIFE BACK!!!

When I got home, I stuffed some cloth in the gap so if this happens again, me being somewhat klutzy, the phone will not slip down like that.

When my son gets up, I will tell him that he was right, and I was wrong, and I don’t even care because I have my phone back, and besides, you have to admit when you’re wrong like that.

 

Share

When Worlds Collide! Kapow! Writing and Database Design

Sunday, February 14th, 2016

I continue to marvel at the intersection of my writing life and my tech life. One of my job functions is to architect databases that will hold the information that runs websites or pays bills or, report on the information in the database. It’s remarkably easy to get wrong. Because I’m a data architect and not a software developer, my approach is always, how would I set this up so we meet the users needs and protect the data?

Every software dev is now slapping her/his forehead and groaning.

Dude. We’ll negotiate the joins later, OK?

But no. We’re not doing it your way because your way is fast on the app side and 30 days later we can’t report on the data. Or it’s not actually what the user needs.

So. Authors who write series, often more than one, need a way to show readers the order of the books in the series. Sounds simple, right? Let the author number the books from 1 through a bazillion.

(And now all the authors are now groaning, going, wait! What about prequels, novels and novellas? where I didn’t write them in chronological order or where readers want Novel order interspersed with Novellas … without having to number Book 7 in the series 9th in the order because there are intervening novellas or short stories.)

Readers are doing much the same.

I know, I know. And I’m so sorry.

surprise/not surprised

I’m surprise/not surprised that ALL the book vendors have gotten the design of this wrong. It SOUNDS easy, but it’s not. We all see the huge disconnect between all the vendors who went “Let them number the books IN WHOLE NUBMERS!” and what authors and readers actually need.

A structured database, like SQL Server or Oracle could do this of course. So could a NoSQL db, but either way you still have to understand this isn’t just a numbering problem. NoSQL probably makes this easier to achieve.

But is OBVIOUS no one at the vendors actually talked to enough authors or even publishers about this. If they had, they’d know they were about to design it wrong.

Painful Lessons of Bad Data Architecture

One of the early lessons for any database person is that when the structure doesn’t provide users a way to do what is needed in real life, they will find workarounds that fuck up your data.

Like when there’s no field to hold (easy example) a middle name and someone goes by their middle name. Users will enter the middle name in the first name field OR the last name field, or a field for Company name. And then you are fucked because you have no way easy way to know that Roberta Ellen Smith is in your database as

firstname: Roberta Ellen
LastName: Smith
CompanyName: Acme Widgets

AND/Or
firstname: Roberta
LastName: Ellen Smith
CompanyName: Acme Widgets

AND/OR
firstname: Roberta
LastName: Ellen
CompanyName: Smith/Acme Widgets

Now get me a list of all the people whose last name is Smith.
Oops.

I’m not making this up, except the names. I’ve seen databases like this.

Real World Reading and Writing

In the real world of reading and authoring books, a series can have:
Novels
Novellas
Short stories
Something else we didn’t think of yet.

And the order might go like this:

Book 1
Book 2
Novella 1
Book 3
Short Story 1
Book 4
Novella 2

AND it’s possible the author wrote these stories in this order

Short Story 1
Book 2
Book 3
Book 4
Book 1
Novella 2
Novella 1

So that by the time they’ve written novella 1, which in the chronology of events in the SERIES places Novella 1 chronologically before Book 3, they will need to reorder their books.

How do I know this is possible? Because I’m a data architect and I’ve seen stuff like this before but also because here’s the story series order of my Sinclair Series books:

Books in chronological order according to the World
Mary’s story
Anne’s Story
Lucy’s Story
Emily’s Story

Here’s the order I have written or will write them in:

Anne’s Story
Lucy’s Story
Emily’s Story
Mary story

Why? Because PUBLISHING doesn’t commit to series and sometimes neither do authors. A book that was intended as a stand alone turns out to have characters in it readers are dying to read about, and some of those characters’ stories would be BEFORE the time setting of the book that was published first.

Authors commonly want to number books like this:

Prequel (Book 0)
Book 1
Novella 1.5
Book 2
Book 3
Novella 3.5

Which is a fairly obvious solution from their point of view, but with clear issues on the backend if you only allow the number field to accept whole numbers which everyone except All Romance eBooks does.

That’s the problem with the simplistic notion that Books are ordered 1, 2, 3, 4 . . . . N

Authors and Readers – in the same leaky Boat

Readers might want to read the books, regardless of length or nomenclature (novel, novella etc.) in the order they were written. Or chronologically according to events in the series. Or in some order the author has provided for the series.

So, on the backend, book order is actually considerably more fluid than any vendor currently provides to authors who would like to number their books so readers can figure out which stories they want to read first.

So vendors end up giving us this, which is completely inadequate for authors and readers alike.
Book 1 = Book 1
Book 2 = Book 2
Book 3 = Novella 1
Book 4 = Book 3
Book 5 = Short Story 1
Book 6 = Book 4
Book 7 = Novella 2
Book 8 = Book 5

Because to get the books listed in reading order authors have to say Book 5 is Book 8 when it isn’t book 8.

Obviously, you should allow the author to provide a reading order that does not link reading order to nomenclature.

1 = Book 1
2 = Book 2
3 = Novella 1
4 = Book 3
5 = Short Story 1
6 = Book 4
7 = Novella 2
8 = Book 5

So, you show the end user this:

Vampire Flowers Series – Reading Order

Flowers in My Garden, prequel
Flowers in my Sock drawer, Book 1
The Gnome Attack, Novella 1
Flowers in my mailbox, Book 2
Gnome Attack!!, A short story
Bees Knees, Novella 2

Reading Order, as a numbering concept unrelated to any other order, would also permit a reader to get a list of just the novels, in reading order.

But nobody asked me.

Share

Amazon Reviews and Timestamps

Sunday, January 31st, 2016

Recently, I followed a twitter link that landed me here: k-lytics.com which page is a tutorial for authors about how to understand Amazon links and lessen the risks of review removal. Although the main take away, which is it’s better to use a link that contains only the ASIN, the tutorial is wrong on just about every technical fact.

It’s hard to know where to start when everything is so . . . wrong.

Note: I’ve gone in and clarified when I’ve realized that I wasn’t specific enough or I used language than means something different to a developer or database admin than it likely does to a person who isn’t either of those things.

The TL;DR

Amazon URLs don’t identify the person who did the search so Amazon is not using incoming links as a criteria for review removal. The value of qid= in a URL does not assist in distinguishing the user account. While there are reasons to use a “clean” Amazon URL in your links, identification of your Amazon account as the link source is not one of them.

Credentials

Before I continue, and on the off chance that someone who does not know my background reads this post, here’s a statement of my technical credentials:

I am a former web developer. I have worked in dev-ops. (Technically, I think I still do, but for a much smaller company without scheduled product release cycles.) I am a SQL Server DBA and data architect. It has been my job to design and maintain the database back-end for commercial, enterprise web applications. I have attended daily meetings with software architects and developers where my responsibility was to head off boneheaded code and bad database designs or to design such structures for them. My current job is with a much smaller company, but the skill set is still required.

When someone starts talking about interpreting URLS and particularly about databases, this is squarely in my technical expertise. Especially the database stuff.

The Actual Problem

Amazon has identified relationships between the poster of a review and the creator/seller of the product as reasons they will remove reviews. The exact words (see quote infra) are “perceived to have a close personal relationship” or “a direct or indirect financial interest.”

In order to establish these things, Amazon has to connect a given Amazon account with one or more external accounts. More on that later.

And so, you might think, of course a link on a third party site is an external thing that might create the appearance of a relationship. But link clicks would be a remarkably inefficient way of deriving that information.

A link, sitting on your website, or facebook, or twitter, or pinterest, gets clicked on by someone and that someone ends up at Amazon and they buy your book. This is not behavior that Amazon, or anyone who sells stuff on the web wants to discourage.

We know that Amazon has removed reviews from readers who love an author and post reviews of every book that author writes. That is because they have identified that the reader has done something such as like the author on Facebook, and that there is, therefore, an outside, personal, relationship between the reader and the author. Facebook, as do other social media companies, provides a wealth of information about who likes what/whom. That information is pretty easy to find out. The contents of a link URL are irrelevant to that determination.

That relationship is NOT contained in the URL.

If you have an Amazon link on your Facebook page and someone clicks on it and buys your book, the smoking gun isn’t the URL string of the link. It’s the referrer information that tells amazon that the click came from your Facebook page or profile (And if it’s from your profile rather than your author page, then you are likely asking for a false positive).

An Amazon URL is a Dumb and Inefficient Way to Infer Relationships

Amazon is unlikely to be using URL strings from incoming third parties (your website, facebook etc.) to figure out which reviews are suspect. They surely are interested in incoming links, but not as implied in that article. Parsing URLS for such information would be a strange and inefficient way to get that information, especially when third parties make it easy to mine far more relevant data.

Carolyn’s Theory

Personally, my theory is that authors who are using the same email address for Amazon, writing- related social media activities, and their personal lives, are more likely to run into problems with Amazon incorrectly deriving personal relationships where none exist. I suspect that logging in all over the place with a Facebook account only exacerbates the issue. Using the same email address at Amazon, Facebook, Twitter, etc, or filling in alternate email contacts at those sites where that alternate email is the same as your Amazon email account are going to make it really easy for Amazon to find connections and come up with real, or incorrect, derivations of actual problematic relationships. Especially if you haven’t locked down your privacy settings.

The Technical Problems with the Analysis

Right. So, the claim is that a link containing stuff besides the ASIN is sufficient to invoke a review removal.

Amazon knows if a reviewer bought the item they’re reviewing. And they surely know what events led to the reviewer’s “buy this” click. If it’s a link from a third party site, then they have whatever information is in the referring link. However, that link does not contain the account information of the person who copied and pasted the link.

The tutorial implies that a qid value, should it exist, is sufficient to identify the account that created the link. That is false.

The qid value is an Epoch timestamp — the number of seconds since January 1, 1970. This value is precise only to the second. That right there tells you there’s a huge problem with the analysis in the tutorial.

The idea that the qid value provides enough information to identify the account that made the search is just … wrong.

The addition of the qid value does not and cannot guarantee uniqueness of the string. It is possible for two people make the exact same search at the exact same second and click on the top result at the same second. In such a case those URLs will be identical.

How a DBA Gets Fired

Uniqueness is a key component of database design. If the data architect gets this wrong because they fail to account for the possibility of collisions where two objects cannot be distinguished from each other, they’re going to be out of a job.

Unique Snowflakes MUST Exist

It is actually impossible to have uniqueness on a timestamp that is precise only to the second. I imagine many people unfamiliar with such concepts think that precision to the second is pretty darn precise. But in this context, it is not. It’s also not precise enough for things like the Olympics, by the way.

When the ability to uniquely identify something is required, you don’t choose imprecise values to achieve that.

Frankly, this is a dumb discussion. If you want to track search queries by account this isn’t how you do it.

The qid does have a useful purpose, but it’s not identifying the user who made the search.

Let me remind you that when Amazon needs to know what user account referred an incoming link, they don’t say, “No worries, we have that in every URL!” What they say is, sign up for an associates account so we can give you a uniquely identifying string that tells us the link came from you.

More Problems

The tutorial goes on to state that the number of times the link was clicked on provides evidence of author manipulation. No. Mere clicks on a link are evidence of popularity of the content and the popularity of the author. If number of clicks alone was evidence of manipulation then popular authors would disproportionately suffer from such a system. Further, if that were true, then no author should ever use associates links.

Additional information is needed in order to infer manipulation and that information is not in an Amazon URL.

I think it’s pretty ridiculous to think that Amazon would take punitive actions based on data that does not identify the account that made the link. The implication that it’s the qid portion of the URL, is, in a word, bullshit.

Here’s what Amazon says about its policy (found here):

Authors and artists can add a unique perspective and we very much welcome their customer reviews. While we encourage reviewers to share their enthusiasm and experience, there can be a fine line between that and the use of customer reviews as product promotion. We don’t allow anyone to write customer reviews as a form of promotion and if we find evidence that a customer was paid for a review, we’ll remove it. If you have a direct or indirect financial interest in a product, or perceived to have a close personal relationship with its author or artist, we’ll likely remove your review. We don’t allow authors to submit customer reviews on their own books even when they disclose their identity.

And here’s a few of the items that prompt removal:

  • A product manufacturer posts a review of their own product, posing as an unbiased shopper
  • A customer posts a review in exchange for $5
  • A family member of the product creator posts a five-star customer review to help boost sales
  • An artist posts a positive review on a peer’s album in exchange for receiving a positive review from them

For that last one, substitute “author” for “artist” and “book” for “album.”

There’s very, very little in any Amazon URL that provides any of that information.

It’s not the purchase that is suspect. Amazon knows who bought what. Amazon is saying there is a non-commercial, personal relationship between the poster of a review and the author.  The URL doesn’t provide a smoking gun of “These people are buddies outside this commercial transaction!”

What people suspect Amazon is doing for the purposes of determining those relationships is examining things like connections between Amazon accounts (Kindle sharing, mailing addresses, etc) or links between Amazon email addresses and possibly IP address that indicate that one person is posting under multiple identities.  They’re also believed to be looking at other social media accounts, including Facebook and places where unwise authors might obtain insincere reviews, such as Fiverr, including taking legal action against those services. Gifting a book to a reader is something that appears to trigger an issue with a subsequent review.

Even More Problems

If you listen to that tutorial, you’ll come away thinking several incorrect things.

The tutorial implies that the qid, which is a Unix Epoch timestamp (the number of seconds since January 1, 1970) is a unique identifier. This is so false I immediately lost track of the tutorial because I was all wha??? (No worries! I listened three times to get their statements straight.) It manages to also imply that the qid somehow identifies the user making the query. That is also false.

It makes a big deal of demonstrating that a qid value changes over time. Um, doh?

Wrong about Short Links, Too

Then the tutorial talks about short links and it implies that using a short link will strip the identifying data from a copied Amazon URL. That, too, is false. Whatever is contained in the source URL that you paste into your short link destination will be used to resolve the destination of the click.

So, suppose you use bit.ly/mybook  as the link you post at FB.

When someone clicks on your FB bit.ly link here is what happens:

The user goes along for the ride to bit.ly where bit.ly looks up the destination you gave it for bit.ly/mybook (this happens really quickly. The user is unlikely, but only unlikely, to notice the millisecond or so that they’re at bit.ly.)

Bit.ly sends the user to the destination you copied and pasted from Amazon. The ENTIRE URL you copied and pasted. Including any applicable qid or other search string.

Lastly, the tutorial completely omits consideration of the use of Amazon associates links. If it’s true that Amazon is using information from incoming third party links to figure out whose reviews to remove, then authors should NEVER use associate links. An associates link actually DOES identify the source of the user account that made the link. But that’s an absurd result. Amazon wants people to use their associates links.

Precision to Websites and Databases

Amazon processes millions of transactions and there are, guaranteed, many many queries that occur in the exact same second. Database systems that need to know which transaction to commit first are looking at milliseconds and nanoseconds. Therefore, a timestamp that is precise only to the second is inadequate for the identification of separate transactions. An Epoch timestamp might uniquify, but it cannot uniquely identify. And, even if it were used to add some value to a search string to make it unique, an imprecise value like that would not guarantee there would not be a collision.

Here’s what the timestamp can efficiently do: create an easy, lightweight way to compare the start time of the product search result to actions taken later. So you know something like, how long it took the user to click buy. It’s easy and lightweight because all you have to do is some arithmetic like subtract one epoch value from another.

Why You’d Want a Clean URL

Long URLS are subject to errors that break the link. Certain characters, such as spaces and ampersands, may need to be encoded so the URL is correctly parsed. You might not get the entire URL. It’s a lot of work. It’s easier to read your html and other analytics.

But it’s not because Amazon is using a qid to identify the person who created the link.

Share

Mid-Month Report in which I talk about JSON

Wednesday, January 20th, 2016

So far, my 2016 has been a success factory!

1. Operation New (to me) Desk remains on target, if not slightly ahead of schedule.
2. My keyboard tray arrived and I assembled and attached it.
3. My file cabinet/printer stand arrived, and I assembled it, put it next to the desk, put the printer on it, and then put stuff in the drawers.
4. I researched, decided, ordered and obtained a new chair. It arrived and I assembled it and I’m now sitting on it.
5. My Demon Warlord is at the proofreader.
6. I got all the files for the stand alone novella release of An Unsuitable Duchess to the person I decided to use for all my formatting going forward.
7. I asked my graphics guy to do color versions of some custom work he did for me because I’ve stopped kidding myself that I want to deal with it myself. Finals arrived today. They’re awesome.
8. I found someone to do the cover for An Unsuitable Duchess and that’s done. I’m just waiting for the final files.
9. I hire someone to do a software update for me because that’s another thing I don’t have time for.
10. Print cover for My Demon Warlord is done and awaiting final page count.

And, here’s a coincidence that I thought was pretty funny, which in a way is No. 11 only it’s not done yet. I was emailing with my formatter who mentioned that he was using JSON files for a database source and I sent him this screenshot:

screenshot of a JSON file. Code.

JSON

Because I was, at at very moment, working out the schema for my planned DB to manage my ebook info and links because I didn’t put it all in something SQL Server-ish because — I do that all day, and I didn’t want to come home and do more even though I could have bothered to learn MySQL and used that but to be honest, a NoSQL solution was way more interesting to me and I even think it’s more appropriate, because NoSQL is kind of designed for systems that are a bit fluid the way this book shit is, and now I don’t have deal with NULLs and spend more time than I want to properly architect the relational version because, quite literally, I would be sitting there going, but how would I scale this out?? if I took shortcuts that denormalized the tables. That would drive me nuts. And so.

I picked one book as my sample schema document and started setting out the data and mapping the arrays and since I have mongoDB installed on the macs I guess I’ll just put it all in MongoDB and now I’ll get my money’s worth from the time I decided to buy my own Mongo GUI tools from when I was playing around with MongoDB at the previous day job and getting SQL Server to produce JSON files for me. Supposedly SQL Server 2016 CTP 3.2.1 (which I have at the current day job)  can supposedly do this without all the shennanigans I went through in SQL 2012. So, fun, eh?

 

 

Share

Operation New Desk: Sub-Optimal Areas of Organization

Sunday, January 10th, 2016

To Avoid Hell, Organization Requires Commitment

So, some time ago, the day job negotiated renovated office space… and a lot of office equipment was offered up to employees. I snagged an adjustable desk because I want to set up a treadmill desk. I measured etc and confirmed the desk would fit in my room. I found a college student with a truck and arranged to get the desk to my house. The desk isn’t huge but it’s not light. But also, I was in the middle of trying to finish My Demon Warlord and Seduction in Winter and so I ended up having the guy park the desk in the living room until I had time to clear out drawers etc…

And there the desk stayed. For a really really long time. Because desk drawers. Because all the stuff I had on the desk… And the closet and and…. And every spare minute needed to go into writing.  Organization requires commitment of heart and time. If you stop halfway into an organization project you might as well go live in hell.

I was starting to hate the desk, taking up space in the living room on its end… A too-inviting target for small dogs.

And I was dreading the work…

But then I finished My Demon Warlord and sent it to the copy editor and whoo-hoo! My son was still home for Winter break and I finally had time to spend with him. I had him set up Apple TV when he got here and then signed up for the free trial of Netflix. And he had been watching Netflix. With my break between books he and I watched the first episodes of Jessica Jones, Daredevil, and Sense8. Then we watched several episodes of Jessica Jones and then he went back to school.

Sub-Optimal Areas of Organization

And so, there I was with a long New Year weekend and a desk taking up space and so began Operation New Desk Binge Watch. I emptied desk drawers and surfaces and other places of  . . . let’s be kind and call them “sub-optimal areas of organization” and I brought out the shredder and put all this “stuff” into boxes and watched Jessica Jones while I shredded and sorted and discarded and found new homes for things. Rinse and repeat. When I ran out of Jessica Jones, I started on Daredevil.

Then I was exhausted. But yesterday, I looked at my office with my old icky desk cleaned off but still with a broken drawer and said “I’m going in!”

Only Slightly Bruised

I got the old desk out by myself because it’s not all that heavy, just 20 years old and, well, it’s had a broken drawer for years and no holes for cables so I wasn’t even sorry to see it go out. My brother was over, and he helped me maneuver the “new” desk in. My right toenail is only slightly bruised, I swear.

Then I set up the computer and dropped the cables through the cable holes in the back and Whoo-hoo!!

Phase Two Point Five

So it turns out I need a keyboard tray for the desk, which whatever. I ordered one on Amazon last night — while I was in bed with the lights off. THIS is how technology improves our lives. Since the new desk does not have drawers, today I’m going to buy something suitable that the printer can sit on (currently it’s on the floor) with drawers so I can proceed to Phase 3.

Phase Three

Phase three is to go through all the stuff I didn’t discard and figure out where to store it. Some will go in the Phase 2.5 pedestal thingee and the rest will be relocated to …. somewhere else.

Because of the NSA case, I have a lot of documents that need to be preserved, which is an additional challenge but might as well have it all in the same box, right?

Verdict

Jessica Jones is an awesome series. I love her so hard. I didn’t love Daredevil until about 4 episodes in but then I did. I’m on Sense8 now and I’m blown away at how great this series is. Normally, when I see TV it’s something from the more traditional channels and I watch maybe 2 times a year. But these shows have women doing awesome things and people of all colors and no one is being safe about writing, especially Jessica Jones. Though, a couple of times on Daredevil I thought some of the dialogue was pretty clunky and maybe they needed to call a lawyer a couple of times to get a grounding in how the law might actually work. I’m only a few episodes into Sense8 but yay for a show that isn’t just straight white guys!

Share