The http version of the crossword now serves up a popup with class="xwd__modal--body xwd__start-modal" and a button to proceed with class="pz-moment__button". I believe those have to be acted upon before being able to access the puzzle information.

At the current moment, it appears that .puz generation of NYT is broken.

Today's (Oct. 10, 2021) Sunday puzzle contains clues like:

4 Across: The universe has an estimated 10<sup>82 </sup> of them 23 Across: <i>Field of Dreams</i>

As of now, any HTML contained in clues is passed through unchanged. How this is handled on the user side depends on the client. Black Ink doesn't process any HTML tags, and shows them verbatim to the user. downforacross.com shows the <sup> tag verbatim but applies the <i></i> tag to the rendered clue, yielding italicized text.

How to handle this is a matter of debate. The worst solution IMHO is removing all HTML because that can render clues meaningless; see how 4 Across is handled here. It may be possible to convert certain tags to sensible Latin-1 equivalents; for example the above clue could be rendered as The universe has an estimated 10^82 of them, although whether that is clearer to the average person is debatable. Yet a final option is to leave it as-is, which seems reasonable as long as HTML doesn't appear too often in clues.

I assume it's a false positive, but just a heads up that the default Windows Defender settings detect a virus and deletes the .exe:

https://www.microsoft.com/en-us/wdsi/threats/malware-encyclopedia-description?name=Trojan%3aScript%2fWacatac.B!ml&threatid=2147735503

However, VirusTotal shows only 3 vendors flagging it.

Probably not a lot you can do about it (I'm guessing it doesn't like scraping browser cookies?), but maybe add a note on the Readme so people aren't caught off guard?

Today's puzzle is being parsed so that the grid is incorrect, which in turn makes the grid numbering and clue placement incorrect.

In the following grid answers, extra single black squares are being inserted (numberings are from the original NYT puzzle):

• 28 across
• 36 across
• 41 across
• 85 across
• 96 across
• 100 across
• 133 across
• 138 across

The issue may be related to the puzzle's note: "After you've finished solving, look for an appropriate bonus phrase."

Today's (Sep. 23, 2021) puzzle has a Unicode character (em dash, Unicode '\u2014') in some of the clues, which causes puz.py to throw an exception because the character is outside of Latin-1.

There are two ways to handle this. Method 1 is this pull request, which substitutes sensible Latin-1 alternatives for non-Latin-1 Unicode characters.

The second method would be to output puz files in the v2 format, which supports UTF-8 encodings. The version of puzpy you're using supports this. I haven't investigated how widely-supported the v2 format is; I'll do some further research. In the meantime this pull request addresses the issue.

https://github.com/Q726kbXuN/nytxw_puz/blob/master/nyt.py#L308

Puzzles with circle markups don't get output properly, markup.save() needs to get called to save the markups to the puz extension. E.g. The 2020-10-12 puzzle

This change checks if the puzzle contains any solving notes, and adds them to the output's 'notes' field.

Solving notes are not uncommon in themed puzzles. Example: nyt.py Chrome https://www.nytimes.com/crosswords/game/daily/2008/09/11 Sep1108.puz

Note about notes: When the NYT was publishing in .puz format, they would often put notes into the puzzle title. For example here is their own published version of the above Sep 11, 2008 puzzle. (In the JSON version this actually contains two notes; in the published .puz file one ended up in the title and the other in the notes field.) I'm guessing they did this for compatibility in case software ignored the notes field. However the solvers I've tested (macOS Black Ink and downforacross.com) both handle notes properly so I'm disinclined to put them in the title.

Detects if there are any marked up (circled) squares in the puzzle and adds them to the output file.

Recent example: nyt.py Chrome https://www.nytimes.com/crosswords/game/daily/2021/09/12 Sep1221.puz

Great list @Q726kbXuN that you compiled at glyph_counts.txt. Very interesting to see the frequency of unusual characters in NYT crosswords.

I dropped your list as-is into the code, but removed:

• Glyphs from u0000-u00ff inclusive, since these are valid Latin-1, and
• Emojis, since the encoder's replacements like \N{WINKING FACE} are as good as anything I could come up with.

So this is apparently every non-Latin-1, non-emoji character ever used in a NYT puzzle. A value of 'None' in the table causes the encoder to insert its own replacement. I put in conversions where I could think of something decent, but clearly this is more art than science so please feel free to edit. :)

Background: Because of the way we solve puzzles, I liked the way titles were done in the old NYT .puz files because they included the date and day of week in the title.

These changes reproduce the title, author, and copyright information in the original .puz files published by NYT.

Also, big thanks for this code! It works well and is a lifesaver.

Solution grid includes blank characters without an 'answer' dict element, but with a multi-value 'moreAnswers' array containing:

 ["NOTHING", "EMPTY", "BLANK", "X", "?", "-"]

If we see 'moreAnswers' without 'answers', look for and return the first single-character entry in the array. If none, return 'X'

Signed-off-by: Ed Santiago [email protected]

In the puzzle json, clues with any formatting (italics is the one I see the most) have a "formatted" field in addition to "plain" under their "text" attribute in the "clues" list.

E.g. 21A from December 26, 2021:

{
  "cells":[
      36,
      37,
      38,
      39,
      40,
      41
   ],
   "direction":"Across",
   "label":"21",
   "text":[
      {
         "formatted":"<i>Malice, more formally</i>",
         "plain":"Malice, more formally"
      }
   ]
}

Only the "plain" field is ever used, even though latin1ify function seems to handle HTML tags for this purpose.

https://github.com/Q726kbXuN/nytxw_puz/commit/ac4c7a72c94c0580302b009b97c1b415da8e9ac7#diff-489afda12299c7df1e4831871e50efb4251e75dc0b31d4c662ba56f0c806ba3eR427

Right now the Windows "release" is just a zip up of the output of py2exe. Really, it should be a proper installer, complete with some mechanism to make updates as painless as is possible.

Right now the main workflow is via a console window.

This is OK, but for the less technically inclined, this is a surprising hurdle. I'd like to see a simple GUI that can walk the user through picking a browser for the cookies, validating the URL, creating the .puz file, and as a bonus, launching it if their PC has a handler for .puz files.

Whatever it looks like, I'm OK with the GUI being Windows only, I don't want to drop the the two CLI modes that currently exist, nor break it working on other platforms.