Rich Megginson's Journal
[Most Recent Entries]
[Calendar View]
[Friends]
Below are the 20 most recent journal entries recorded in
Rich Megginson's LiveJournal:
[ << Previous 20 ]
| Wednesday, November 4th, 2009 | | 9:39 am |
389 1.2.4, Windows PassSync 1.1.2
There is a new testing release of 389 Directory Server - 1.2.4 - the new package is 389-ds-base-1.2.4. This new package has one new feature - support for Salted MD5 (SMD5) passwords, and a couple of bug fixes. There is a new version of Windows Password Sync - 1.1.2 - this version supports Windows Server 2008 and 2003, both 64-bit and 32-bit. The release notes are here: http://port389.org/wiki/Release_Notes Current Music: "Ordinary Pain" - Stevie Wonder | | Thursday, August 27th, 2009 | | 8:00 am |
Announcing 389 Directory Server 1.2.2
Looks like we pulled the trigger a little bit too early on 1.2.1. There were several bugs ranging from very annoying to "this crap doesn't work". 389 1.2.2 should address these issues. We apologize for any inconvenience caused by 1.2.1. Details here - http://port389.org/wiki/Release_Notes Current Music: _Psychic Maps_ - Dysrhythmia | | Monday, August 17th, 2009 | | 10:20 am |
Announcing 389 Directory Server 1.2.1
389 Directory Server 1.2.1 was released today. This release includes several new features and bug fixes. This is the first release of the 389 branded server which replaces Fedora Directory Server. Details here - http://port389.org/wiki/Release_Notes Current Music: "Pearls and Transformation" - Arcana | | Thursday, May 7th, 2009 | | 9:27 am |
The Fedora Directory Server Project is now "389"
The Fedora Directory Server Project is now called "389". The details are here: http://directory.fedoraproject.org/wiki/389_Change_FAQ The new project website is http://port389.org (which is currently just an alias for directory.fedoraproject.org) The new IRC channel is #389 We have created aliases for the mailing lists - so 389-users, 389-announce, etc. We're still in the process of rebranding, re-skinning the web site, etc. In the coming weeks you will see new packages with the 389 branding. Everything else is the same - the team, our mission, only the name has changed. We apologize if this change is disconcerting to some of you, we thank your for your support, and we hope to continue to make the 389 project a success. Current Music: unknown artist - unknown track | | Monday, May 4th, 2009 | | 7:19 pm |
Red Hat Directory Server 8.1 Is Released!
The Red Hat Directory Server team is pleased to announce the release of Red Hat Directory Server 8.1. This update to Directory Server 8 contains over 200 bug fixes as well as a number of feature enhancements to further improve the security and functionality of the product. Some of the enhancements were existing customer requests while others have been in community Fedora Directory Server for some time and are now supported in Red Hat Directory Server. Here are some of the highlights of Red Hat Directory Server 8.1 improvements * Enhanced Server to Server Connections with added SASL Kerberos and Start TLS support for connections, including replication, chaining, Windows synchronization, and pass-through authentication. Configuration attributes have been added or expanded to enable the new server connection types. * Enabled dynamic schema reloads. Custom schema files can be added to an instance and loaded on the fly by running a schema reload task. * Added supported for Unix sockets (LDAPI) and autobind. With this update, Directory Server's LDAP operations can run over Unix sockets. Enabling LDAPI also allows the Directory Server to use autobind to authenticate logged in Unix users to the Directory Server automatically, based on their Unix credentials. * New Plug-ins to Simplify Group Membership Management and to Automatically Number Attributes * New Support for 64-Bit Integers for Performance Counters even on 32-Bit Systems * Improved Controls for Handling of Unauthenticated Binds Current Music: "Into the Light" - Joe Satriani | | Friday, May 1st, 2009 | | 10:44 am |
Converting CVS to git
I've been looking at git lately. There seem to be two methods for converting CVS to git. Both require access to the raw ,v files, which usually entails having access to the actual CVS repo files, either directly or via rsync. Both methods also require that you create a mapping that maps the CVS username into the git style Common Name <email address>. So you may have to do some leg work first to track down some info. The first method is using git-cvsimport which in turn uses cvsps to convert CVS updates to patchsets which are then imported into git. This method works fairly well except that it does not import empty tags and branches, only tags and branches which have actual commits. I do most of my development on RHEL5, so I use EPEL5 to get git, git-cvs (for git-cvsimport) and cvsps. The second method uses cvs2git, part of the cvs2svn suite for converting CVS repos to SVN and other SCMs (such as mercurial/hg). The code is available via SVN: svn co http://cvs2svn.tigris.org/svn/cvs2svn/tr unk This is implemented as a python script so python is required too. Step 1 - create your cvs2git.options file - just copy the boilerplate cvs2git-example.options Step 2 - run cvs2git - this will create two files cvs2git-tmp/git-blob.dat and cvs2git-tmp/git-dump.dat Step 3 - create an empty git repo mkdir proj.git cd proj.git git init Step 4 - use git-fastimport to import the blobs cd proj.git cat ../cvs2git-tmp/git-blob.dat ../cvs2git-tmp/git-dump.dat | git fast-import This method did import all tags and branches, including empty ones. One weird thing was that the proj.git directory was initially empty after the fast import - I guess it had to rebuild the indexes. But when I did a "git checkout master" all of the files showed up. I'm not sure if the latter method can be used for incremental sync, so I'll probably go with the second method to create the initial git repo, and use the first method to sync incremental changes made to the CVS repos during the migration phase. Current Music: "Coily" - Ozric Tentacles | | Thursday, April 30th, 2009 | | 4:12 pm |
Editing mediawiki wikis with command line
Sometimes it is nice to be able to use a real text editor to edit wiki pages, or to batch update wiki pages. http://en.wikipedia.org/wiki/Wikipedia:Syntax_highlighting (misnamed?) lists many options for using different editors and batch modes. I first looked at mvs provided with perl WWW::Mediawiki::Client. This did not work for me at all. I got numerous errors and unhandled exceptions. After several unsuccessful attempts to hack it to life, I gave up. Perhaps I got a bum version, or more likely it did not support the mediawiki version I'm using. Next, I tried http://meta.wikimedia.org/wiki/pywikipedia, which is a collection of python scripts used to create wiki bots. These scripts are also useful for general purpose wiki editing. It's a bit harder to use - it seems designed specifically for use with well known wiki sites such as wikipedia.org. But with a little hacking, it can be made to work with just about any site that uses mediawiki. Step 1 - grab the source using SVN
svn checkout --ignore-externals http://svn.wikimedia.org/svnroot/pywikipedia/trunk/pywikipedia/ pywikipedia
Step 2- you have to create a "family" file in which you specify your site and other information specific to your wiki site:
cd pywikipedia/families
cp README-family.txt yoursiteid_family.py
then edit yoursiteid_family.py - the file has boilerplate text you have to fill in. It's very confusing that host nor site nor URL is mentioned. Step 3 - create your user-config.py
cd pywikipedia
edit user-config.py
The pywikipedia site gives instructions. I used something like this:
mylang = "en"
family = 'yoursiteid'
usernames['yoursiteid']['en'] = "yourusername"
editor = 'emacsclient'
where the family must correspond to yoursiteid you used when you created the family file, and yourusername is your user name on the site. editor is optional - this is the editor editarticle.py will use. I suppose you could use vim or gedit or whatever. Step 4 - python login.py - this will prompt you for your password Step 5 - python editarticle.py ArticleName - this will grab the wiki page and put you in your favorite editor - if you make changes and save, editarticle will prompt you with
What did you change?
You should type in some sort of message here. This is the comment which will be added to the page, so that when you look at the page history, you will have some idea of what changed. Current Music: "Dry Ice" - Electromagnets | | Tuesday, April 14th, 2009 | | 12:00 pm |
| | Thursday, April 9th, 2009 | | 6:27 pm |
How to replace wget and unzip in Windows
Many of our build tools on Windows could use plain old native cmd and nmake were it not for the need to use wget to download components and unzip to unzip them. I discovered how to replace these: * bitsadmin - The Windows Support Tools package, available for most Windows versions, has a command line tool called bitsadmin. This (un)helpfully named command can be used to download files from http sites. Typical use in a batch file or Makefile is like this:
"c:\program files\support tools\bitsadmin" /wrap /transfer jobname /download /priority normal http://site/path/to/file c:\full\path\to\dest\filename
NOTES: * You have to specify the full URL of the file to download - you cannot download recursively or with wildcards * http is the only protocol I could get to work - no ftp or https - it's theoretically possible to get https working - for ftp, use the ftp built-in command * You have to specify the full, absolute path of the destination, including file name - you cannot download to ., or to \destdir, you have to specify the full thing as drive:\full\path\to\destdir\filename - otherwise you will get cryptic, meaningless errors * unzip.vbs - Windows has had built-in support for the zip compression mechanism for quite some time - if you've ever opened a compressed folder in explorer, you've seen this in action. Unfortunately, there is no simple unzip command line tool built-in. Fortunately, Windows has a powerful Visual Basic/JScript scripting language interpreter built-in called cscript. This allows you, through the magic of COM, to access many (not very well documented) Windows internals. Here is some cscript Visual Basic code I use to unzip files:
' get our FSO object
Set objFSO = CreateObject("Scripting.FileSystemObject")
' src zip file is arg 0
SRC = objFSO.GetAbsolutePathName(objArgs(0))
' dest folder is arg 1 or "."
If objArgs.Count < 2 then
DEST = objFSO.GetAbsolutePathName(".")
Else
DEST = objFSO.GetAbsolutePathName(objArgs(1))
End If
' create dest folder if it does not exist
If not objFSO.FolderExists(DEST) Then
objFSO.CreateFolder(DEST)
WScript.Echo "Created new folder", DEST
End If
' see if file ends in .zip - if not (e.g. .jar) make temp copy
' that ends in .zip
Dim newSRC
newSRC = ""
If not Right(SRC, 4) = ".zip" Then
newSRC = SRC & ".zip"
objFSO.CopyFile SRC, newSRC, true
SRC = newSRC
End If
' get the shell application object used to do the unzip
Set objShell = CreateObject("Shell.Application")
Set objSrc = objShell.Namespace(SRC)
Set objDest = objShell.Namespace(DEST)
objDest.CopyHere(objSrc.Items)
' remove temp zip, if any
If Len(newSrc) > 0 Then
objFSO.DeleteFile(newSRC)
End If
WScript.Stdout.Write "Done. Copied contents of " & SRC & " to " & DEST
You run it like this: cscript //nologo file.vbs. The first part is just standard stuff covered in the cscript getting started guide. The real magic happens with Shell.Application. This apparently uses the desktop/explorer directly, just as if you had opened and copied a compressed folder on the desktop. IF you have a large zip file, you can even see the progress window with the flying files. Re-writing the VB script to JScript is left as an exercise to the reader. Current Music: "Mountain Waltz" - Steve Morse Band | | 6:22 pm |
Announcing Fedora Directory Server 1.2.0
We released Fedora Directory Server 1.2.0 this week. This release includes several new features and many, many bug fixes. - remove-ds.pl - remove a directory server instance
- remove-ds-admin.pl - wipe out everything (for use when you run setup-ds-admin.pl and get an error, and just want to start over from scratch)
- New Schema - now includes schema for autofs, samba, and many other apps
Details here - http://directory.fedoraproject.org/wiki/R elease_Notes In addition, we are working on the requirements for the next release, tentatively numbered 1.3.0. We've put together a roadmap with an impossibly large list of features for this release. Fedora DS Version 1.3.0 features - http://directory.fedoraproject.org/wiki/R oadmap Your input is welcome! Current Music: "Teenbeat" - Henry Cow | | Friday, January 9th, 2009 | | 4:32 pm |
Directory Server work
I've just been cranking out bug fixes for directory server. One interesting thing I found out about console debugging with Eclipse - the only way I could get it to work with JSS (HTTPS and LDAPS connections) is to put a symlink to the jss shared lib in the jre bin directory. For some reason, no matter what java.library.dir or LD_LIBRARY_PATH I set, Eclipse would always look for the JSS shared lib in the JRE directories, not in the system directories. I finally had to run strace on the Eclipse java process to figure out what it was doing. On RHEL5 x86_64, using the ibm 1.5.0 jre, you have to do something like this:
/usr/lib/jvm/jre-1.5.0-ibm.x86_64/bin/libjss4.so -> /usr/lib64/libjss4.so
Ugly, yes, but I couldn't figure out any other way to get it to work. I also found out that you can use epochs in RPM Requires and BuildRequires - this is even required to get it to work correctly sometimes. For example, on RHEL5
BuildRequires: java-devel >= 1.6.0
will match java-1.5.0-ibm-devel. I don't know why. But if you add the epoch
BuildRequires: java-devel >= 1:1.6.0
it will correctly find only 1.6 JDKs and not 1.5 or others. Weird. Current Music: "Variato 15" - Glenn Gould - Goldberg Variations | | Friday, November 21st, 2008 | | 2:41 pm |
Console Server-To-Server improvements
I've been working on adding support for StartTLS, SASL/GSSAPI, and SASL/Digest-MD5 to the console for setting up server to server connections. My first stop is replication. To aid my task, I've been using Eclipse. This makes life much, much easier - point and click to find definitions and references, easy refactoring, debugger, etc., etc. Java IDEs have certainly come a long way, especially Eclipse. One nice surprise was that Active Directory (2003 Server) supports the LDAP StartTLS extended operation. All LDAP clients and server should support StartTLS, which is the standard way to use TLS/SSL with LDAP. However, one of the problems with StartTLS is that there is no standard way to specify this in an LDAP URL. Many applications specify all of their LDAP parameters compactly in LDAP URL form instead of having separate parameters for host, port, use ssl, search base, scope, etc. So at the moment all apps have to have a separate parameter for start_tls e.g. nss_ldap uses ssl <on|off|start_tls>. The LDAP URL format is extensible, we just need to define a standard format for this. Current Music: "Search for the Suicide King" - Ohm | | Monday, November 17th, 2008 | | 10:14 am |
Kerberos, startTLS
I've been working on adding support for server to server Kerberos, for example, the ability to use LDAP SASL/GSSAPI for replication. http://directory.fedoraproject.org/wiki/Server_To_Server_ConnOne problem I've found with the LDAP SASL/GSSAPI layer is that there is no way to pass Kerberos context in or out using this API. So the way it works is that you have to use the principal and keytab or password to acquire the credentials, then create a a credentials cache, then set the environment variable KRB5CCNAME to point to that cache. This is definitely not optimal, especially in a multi-threaded server environment, and if you want to be able to support multiple principals. I have not investigated the code very much, but I would assume at some point the SASL/GSSAPI code calls krb5_cc_default(), and there is no way to get a hook here to supply the credentials. SASL does provide a callback mechanism to handle the interactive server request/response phase, but it only allows you to provide a username, not the actual Kerberos credentials. The Kerberos API does, however, provide a cc api. So one possible approach would be to provide a new ccache type e.g. MYTYPE. The server at startup would set KRB5CCNAME=MYTYPE:someid. MYTYPE would work similar to the MEMORY type, storing the credentials in memory. Then, when the GSSAPI code asked for the credentials for this type (and specifying the client and server principals), the ccache code could provide whatever was needed. One problem is that the ccache implementation is only available in k5-int.h, not the public krb5.h interface, so I'm not sure if this would be possible for general purpose code outside of Kerberos or GSSAPI. I've also been changing the server code to use startTLS for TLS/SSL wherever possible, instead of the deprecated LDAPS method. One of the problems here is that there is no standard way to encode the startTLS operation into an LDAP URL - you can specify ldap://host.... or ldaps://host... but there is no way to specify ldap://host/?????startTLS. There was a proposal on one of the openldap lists to add support for this to the openldap API, but no clear consensus, and no one has recently broached the subject on ldapext or umich. So for now, in the server code, I have special attributes or values depending on the context. For example, in the replication code, there was already an attribute whose values where LDAP or SSL - it was a simple matter to add an allowed value TLS. In the chaining code, I had to add another attribute "nsUseStartTLS". One pleasat surprise was in the Windows Sync code - Active Directory (at least with Windows Server 2003) does support startTLS. | | Friday, September 5th, 2008 | | 1:20 pm |
Directory Server admin scripts
I've been asked about Fedora Directory Server management scripts from time to time. I find these very useful for development testing. I started out with NDSAdmin.pm which is based on Mozilla::LDAP, then ported that to use Net::LDAP (NDSAdminNL.pm). A couple of years ago, in an effort to learn python and python-ldap, I ported these to dsadmin.py. I mostly use them for replication testing - I can very quickly set up multiple servers, even on remote machines, configure replication, and test different scenarios. There are a large collection of scripts which use these modules that demonstrate how to set up replication, and how to use the underlying LDAP API. Also, in an effort to learn git, I've made these public over at github: - wiki page - http://github.com/richm/scripts/wikis/home
- source - http://github.com/richm/scripts/tree/master
Current Music: "Jozy" - John McLaughlin Trio, Live at Royal Festival Hall | | Wednesday, May 14th, 2008 | | 12:20 pm |
Fun with bourne shell: Argument list too long
I've been porting an old shell script application (written in a decidedly non-portable ksh variant) to use plain old least common denominator Bourne shell. The application is many, many lines of code with nested "libraries" of functions and variables source'd in. I got the following error: /bin/uname: Argument list too long I looked at the call to uname - uname -s - hmm, no arguments to speak of there. A little searching revealed that Argument list too long can be returned if the _environment_ is too big. So I dumped out env and set just before the call to uname. env.out was a whopping 80kbytes! What in the world was setting all of those env. vars???? I went back to the "main" script. Near the top was this: set -a D'oh! After removing that, the env size dropped back to a manageable 3kb, and now it's working much better. Still have some other porting issues, but this one was a real head scratcher . . . Current Music: "Larks' Tongues in Aspic, Pt. 1" - King Crimson, _The Great Deceiver_ Disc 4 | | Thursday, May 1st, 2008 | | 2:26 pm |
DSML Gateway source is now available
The DSML Gateway is a Java Axis web service that translates DSMLv2 into LDAP. In 7.1 this was bundled with the binary distribution, but was not included with 1.0 or later releases. The new code works with the Fedora DS 1.1 and uses Java, Tomcat, and Axis. We do not have a binary distribution at this time, but the source is pretty easy to build for those folks familiar with web application development. For more information, see http://directory.fedoraproject.org/wiki/DSML_Gateway Current Music: "Not of This Earth" - Joe Satriani | | Friday, April 25th, 2008 | | 2:21 pm |
FreeIPA 1.0 is released!
The FreeIPA team have just released version 1.0. Congrats to the team! I know they been working very hard the past few weeks (and had to deal with some hassles caused by a fedora-ds bug :P ). If you're not familiar with FreeIPA, head on over to http://freeipa.org - If you've ever had problems getting Kerberos to work with or without LDAP, you will probably be interested. And that's only the beginning. Current Music: "I Don't Know What You're Talking About" - Husker Du | | Thursday, April 24th, 2008 | | 3:24 pm |
Fedora DS web apps and security updates
I'm pleased to announce the release of the web applications for Fedora Directory Server 1.1. These are the web applications bundled with the Admin Server component in versions 1.0 and earlier. We have moved them into their own separate package in 1.1 and have made them available in the fedora-ds-dsgw package. The applications are: * phonebook - a simple search and info lookup application * org chart - a graphical view of your organization structure, based on the manager attribute * gateway - a more complex search, view, and edit application, including the ability for "self service", including password changes The applications are tied together, so you can view the org chart for someone you look up in the phonebook, and vice versa. The phonebook and gateway applications use very primitive templates, like very basic jsp or velocity pages, but they at least allow for some customization. The new Gateway Customization guide is not ready yet, but the old one still mostly applies (http://www.redhat.com/docs/manuals/dir-s erver/). In other news, there were two security vulnerabilities in the fedora-ds-admin package recently released: The new package is fedora-ds-admin-1.1.4-1 This package is available from the Fedora yum repository for F-7 and later, or from the dirsrv yum repo on Fedora 6 and EL5. See Install_Guide for information about how to use these yum repositories for your platform. Current Music: "New Day Rising" - Husker Du | | Wednesday, April 2nd, 2008 | | 11:03 am |
Fedora Directory Server news
1) I've finally finished all of the work to port the web apps for use with Fedora DS 1.1: * autotool-ization * FHS-ification * HTML Tidy cleanup * Lots of other cleanup I've submitted this package to Fedora - https://bugzilla.redhat.com/show_bug.cgi?id=440269 - that bug contains links to the sources, etc. I strongly encourage people to use freeIPA as the web based UI for Fedora Directory Server. The current web apps will be deprecated at some point. All new web UI work for User/Group management will most likely go into freeIPA or some other web app based on freeIPA. However, it will be a while before freeIPA has all of the functionality of the current DSGW/phonebook/orgchart tool, so they are still useful for now. The next web app will be the DSML Gateway app. This one is a standard Java web app - I figured it would be better if it were in its own package, rather than being bundled with the others which are CGI based. 2) If you are using Fedora DS 1.1, please update your fedora-idm-console package to 1.1.1 to fix this security problem - https://bugzilla.redhat.com/show_bug.cgi?id=436107 - There is also an update for redhat-idm-console for Red Hat Directory Server customers. 3) Congratulations to the DogTag team! http://pki.fedoraproject.org/ - This is the new site for the open source Certificate System. After much, much wrangling and negotiating, the team was finally able to make the source available, and create a project around it. 4) Work is proceeding on the next release of Fedora Directory Server, which we are tentatively calling version 1.2 - http://directory.fedoraproject.org/wiki/Roadmap - Many of the new features are for freeIPA - http://freeipa.org Current Music: "Pearls and Transformation" - Arcana | | Tuesday, January 15th, 2008 | | 7:47 pm |
|
[ << Previous 20 ]
|
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
richmegginson's journal